Add changes for afd88d8

Contribution/Development.html

@@ -70,6 +70,7 @@
 <li class="toctree-l4"><a class="reference internal" href="#v2-workflow">v2 Workflow</a></li>
 <li class="toctree-l4"><a class="reference internal" href="#v3-workflow">v3 Workflow</a></li>
 <li class="toctree-l4"><a class="reference internal" href="#sr-insects">sr_insects</a></li>
+<li class="toctree-l4"><a class="reference internal" href="#unit-tests">Unit tests</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="#local-installation">Local Installation</a><ul>
@@ -271,6 +272,13 @@ <h3>sr_insects<a class="headerlink" href="#sr-insects" title="Link to this headi
 Both v2 and v3 are supported on the stable branch of sr_insects.  That branch should be
 used to support all development in both versions….</p>
 </section>
+<section id="unit-tests">
+<h3>Unit tests<a class="headerlink" href="#unit-tests" title="Link to this heading"></a></h3>
+<p>The tests/ sub-directory contains a woefully incomplete but growing set of unit tests
+using the <em>pytest</em> framework. These tests are only exercised on Ubuntu 22.04 at the moment.
+consult tests/README.md for how to run them. Unit tests passing (or a very good explanation
+of why they <em>temporarily</em> fail) should be another gate before merging to the main <em>development</em> branch.</p>
+</section>
 </section>
 <section id="local-installation">
 <h2>Local Installation<a class="headerlink" href="#local-installation" title="Link to this heading"></a></h2>
@@ -405,7 +413,14 @@ <h3>Ubuntu 18.04<a class="headerlink" href="#ubuntu-18-04" title="Link to this h
 <p>‘’’</p>
 <p>multipass launch -m 8G bionic</p>
 <p>‘’’</p>
-<p>can run developer tests as per multipass as described above.</p>
+<p>can run developer tests as per multipass as described above.
+Need to edit:</p>
+<ul class="simple">
+<li><p>requirements.txt (remove paramiko and watchdog)</p></li>
+<li><p>setup.py (remove install_requires for paramiko and watchdog)</p></li>
+</ul>
+<p>These packages have comedically complex dependencies. Install them separately
+and they will be usable by sr3, but listing them as a requirement breaks things.</p>
 </section>
 <section id="python-wheel">
 <h3>Python Wheel<a class="headerlink" href="#python-wheel" title="Link to this heading"></a></h3>

Reference/sr3_options.7.html

@@ -1038,6 +1038,14 @@ <h4>logEvents ( default: after_accept,after_work,on_housekeeping )<a class="head
 if the list starts with a plus sign (+) then the selected events are appended to current value.
 A minus signe (-) can be used to remove events from the set.</p>
 </section>
+<section id="logformat-default-asctime-s-levelname-s-name-s-funcname-s-message-s">
+<h4>LogFormat ( default: %(asctime)s [%(levelname)s] %(name)s %(funcName)s %(message)s )<a class="headerlink" href="#logformat-default-asctime-s-levelname-s-name-s-funcname-s-message-s" title="Link to this heading"></a></h4>
+<p>The <em>LogFormat</em> option is passed directly to python logging mechanisms and can be used
+to control what is written to log files.  The format is documented here:</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://docs.python.org/3/library/logging.html#logrecord-attributes">https://docs.python.org/3/library/logging.html#logrecord-attributes</a></p></li>
+</ul>
+</section>
 <section id="loglevel-default-info">
 <h4>logLevel ( default: info )<a class="headerlink" href="#loglevel-default-info" title="Link to this heading"></a></h4>
 <p>The level of logging as expressed by python’s logging. Possible values are :  critical, error, info, warning, debug.</p>

_modules/sarracenia.html

@@ -463,8 +463,8 @@ <h1>Source code for sarracenia</h1><div class="highlight"><pre>
 
 
 <span class="n">known_report_codes</span> <span class="o">=</span> <span class="p">{</span>
-    <span class="mi">201</span><span class="p">:</span>
-    <span class="s2">&quot;Download successful. (variations: Downloaded, Inserted, Published, Copied, or Linked)&quot;</span><span class="p">,</span>
+    <span class="mi">201</span><span class="p">:</span> <span class="s2">&quot;Download successful. (variations: Downloaded, Inserted, Published, Copied, or Linked)&quot;</span><span class="p">,</span>
+    <span class="mi">202</span><span class="p">:</span> <span class="s2">&quot;Accepted. mkdir skipped as it already exists&quot;</span><span class="p">,</span> 
     <span class="mi">203</span><span class="p">:</span> <span class="s2">&quot;Non-Authoritative Information: transformed during download.&quot;</span><span class="p">,</span>
     <span class="mi">205</span><span class="p">:</span>
     <span class="s2">&quot;Reset Content: truncated. File is shorter than originally expected (changed length during transfer) This only arises during multi-part transfers.&quot;</span><span class="p">,</span>
@@ -473,6 +473,7 @@ <h1>Source code for sarracenia</h1><div class="highlight"><pre>
     <span class="s2">&quot;Not modified (Checksum validated, unchanged, so no download resulted.)&quot;</span><span class="p">,</span>
     <span class="mi">307</span><span class="p">:</span> <span class="s2">&quot;Insertion deferred (writing to temporary part file for the moment.)&quot;</span><span class="p">,</span>
     <span class="mi">417</span><span class="p">:</span> <span class="s2">&quot;Expectation Failed: invalid notification message (corrupt headers)&quot;</span><span class="p">,</span>
+    <span class="mi">422</span><span class="p">:</span> <span class="s2">&quot;Unprocessable Content: could not determine path to transfer to&quot;</span><span class="p">,</span>
     <span class="mi">499</span><span class="p">:</span> <span class="s2">&quot;Failure: Not Copied. SFTP/FTP/HTTP download problem&quot;</span><span class="p">,</span>
     <span class="c1">#FIXME : should  not have 503 error code 3 times in a row</span>
     <span class="c1"># 503: &quot;Service unavailable. delete (File removal not currently supported.)&quot;,</span>

_modules/sarracenia/config.html

@@ -1183,11 +1183,10 @@ <h1>Source code for sarracenia.config</h1><div class="highlight"><pre>
             <span class="k">if</span> <span class="n">v</span> <span class="o">==</span> <span class="s1">&#39;None&#39;</span><span class="p">:</span> 
                 <span class="n">sv</span><span class="o">=</span><span class="nb">set</span><span class="p">([])</span>
             <span class="k">else</span><span class="p">:</span>
-                <span class="k">if</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="ow">in</span> <span class="p">[</span> <span class="s1">&#39;+&#39;</span><span class="p">,</span> <span class="s1">&#39;-&#39;</span><span class="p">]:</span>
+                <span class="n">op</span><span class="o">=</span><span class="s1">&#39;r&#39;</span>
+                <span class="k">while</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="ow">in</span> <span class="p">[</span> <span class="s1">&#39;+&#39;</span><span class="p">,</span> <span class="s1">&#39;-&#39;</span><span class="p">]:</span>
                     <span class="n">op</span><span class="o">=</span><span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
                     <span class="n">v</span><span class="o">=</span><span class="n">v</span><span class="p">[</span><span class="mi">1</span><span class="p">:]</span>
-                <span class="k">else</span><span class="p">:</span>
-                    <span class="n">op</span><span class="o">=</span><span class="s1">&#39;r&#39;</span>
 
                 <span class="k">if</span> <span class="s1">&#39;,&#39;</span> <span class="ow">in</span> <span class="n">v</span><span class="p">:</span> 
                     <span class="n">sv</span><span class="o">=</span><span class="nb">set</span><span class="p">(</span><span class="n">v</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;,&#39;</span><span class="p">))</span>
@@ -1198,6 +1197,7 @@ <h1>Source code for sarracenia.config</h1><div class="highlight"><pre>
                     <span class="n">sv</span><span class="o">=</span> <span class="n">old_value</span> <span class="o">|</span> <span class="n">sv</span>
                 <span class="k">elif</span> <span class="n">op</span> <span class="o">==</span> <span class="s1">&#39;-&#39;</span> <span class="p">:</span>
                     <span class="n">sv</span><span class="o">=</span> <span class="n">old_value</span> <span class="o">-</span> <span class="n">sv</span>
+
         <span class="k">return</span> <span class="n">sv</span></div>
 
 

_modules/sarracenia/flowcb/accept/wmotypesuffix.html

@@ -137,6 +137,10 @@ <h1>Source code for sarracenia.flowcb.accept.wmotypesuffix</h1><div class="highl
 
     <span class="k">def</span> <span class="nf">after_accept</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">worklist</span><span class="p">):</span>
         <span class="k">for</span> <span class="n">message</span> <span class="ow">in</span> <span class="n">worklist</span><span class="o">.</span><span class="n">incoming</span><span class="p">:</span>
+
+            <span class="k">if</span> <span class="s1">&#39;fileOp&#39;</span> <span class="ow">in</span> <span class="n">message</span> <span class="ow">and</span> <span class="s1">&#39;directory&#39;</span> <span class="ow">in</span> <span class="n">message</span><span class="p">[</span><span class="s1">&#39;fileOp&#39;</span><span class="p">]:</span>
+                <span class="k">continue</span>
+
             <span class="n">type_suffix</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">__find_type</span><span class="p">(</span><span class="n">message</span><span class="p">[</span><span class="s1">&#39;new_file&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">:</span><span class="mi">2</span><span class="p">])</span>
             <span class="c1">## FIXME confused as to how this could ever be true since find_type never returns &quot;UNKNOWN&quot;</span>
             <span class="c1">#if type_suffix == &#39;UNKNOWN&#39;:</span>
@@ -147,10 +151,17 @@ <h1>Source code for sarracenia.flowcb.accept.wmotypesuffix</h1><div class="highl
                 <span class="k">continue</span>
 
             <span class="n">message</span><span class="p">[</span><span class="s1">&#39;new_file&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">message</span><span class="p">[</span><span class="s1">&#39;new_file&#39;</span><span class="p">]</span> <span class="o">+</span> <span class="n">type_suffix</span>
+
             <span class="k">if</span> <span class="s1">&#39;rename&#39;</span> <span class="ow">in</span> <span class="n">message</span><span class="p">:</span>
-                <span class="n">message</span><span class="p">[</span><span class="s1">&#39;rename&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">message</span><span class="p">[</span><span class="s1">&#39;rename&#39;</span><span class="p">]</span> <span class="o">+</span> <span class="n">type_suffix</span></div>
+                <span class="n">message</span><span class="p">[</span><span class="s1">&#39;rename&#39;</span><span class="p">]</span> <span class="o">+=</span> <span class="n">type_suffix</span>
+
+            <span class="k">if</span> <span class="s1">&#39;fileOp&#39;</span> <span class="ow">in</span> <span class="n">message</span> <span class="ow">and</span> <span class="s1">&#39;rename&#39;</span> <span class="ow">in</span> <span class="n">message</span><span class="p">[</span><span class="s1">&#39;fileOp&#39;</span><span class="p">]:</span>
+                <span class="n">message</span><span class="p">[</span><span class="s1">&#39;fileOp&#39;</span><span class="p">][</span><span class="s1">&#39;rename&#39;</span><span class="p">]</span> <span class="o">+=</span> <span class="n">type_suffix</span></div>
+
 
             <span class="c1"># TODO else -&gt; worklist.rejected.append(message) ?? should this be happening at any point?</span>
+
+
 </pre></div>
 
            </div>

_sources/Contribution/Development.rst.txt

@@ -142,6 +142,15 @@ Both v2 and v3 are supported on the stable branch of sr_insects.  That branch sh
 used to support all development in both versions.... 
 
 
+Unit tests
+~~~~~~~~~~
+
+The tests/ sub-directory contains a woefully incomplete but growing set of unit tests
+using the *pytest* framework. These tests are only exercised on Ubuntu 22.04 at the moment.
+consult tests/README.md for how to run them. Unit tests passing (or a very good explanation
+of why they *temporarily* fail) should be another gate before merging to the main *development* branch.
+
+
 Local Installation
 ------------------
 
@@ -291,6 +300,14 @@ multipass launch -m 8G bionic
 '''
 
 can run developer tests as per multipass as described above.
+Need to edit:
+
+* requirements.txt (remove paramiko and watchdog)
+* setup.py (remove install_requires for paramiko and watchdog)
+
+These packages have comedically complex dependencies. Install them separately
+and they will be usable by sr3, but listing them as a requirement breaks things.
+
 
 Python Wheel
 ~~~~~~~~~~~~

_sources/Reference/sr3_options.7.rst.txt

@@ -1054,6 +1054,15 @@ other values: on_start, on_stop, post, gather, ... etc... It is comma separated,
 if the list starts with a plus sign (+) then the selected events are appended to current value.
 A minus signe (-) can be used to remove events from the set.
 
+LogFormat ( default: %(asctime)s [%(levelname)s] %(name)s %(funcName)s %(message)s )
+------------------------------------------------------------------------------------
+
+The *LogFormat* option is passed directly to python logging mechanisms and can be used
+to control what is written to log files.  The format is documented here:
+
+* https://docs.python.org/3/library/logging.html#logrecord-attributes
+
+
 logLevel ( default: info )
 --------------------------
 

_sources/fr/Contribution/Développement.rst.txt

@@ -127,6 +127,15 @@ Le référentiel sr_insects a sa propre base de données de problèmes, et le tr
 Les versions 2 et 3 sont prises en charge sur la branche principale de sr_insects. Cette branche devrait être
 utilisé pour prendre en charge tout le développement dans les deux versions...
 
+Tests unitaires
+~~~~~~~~~~
+
+Le sous-répertoire tests/ contient un ensemble malheureusement incomplet mais croissant de tests unitaires
+qui utilisent le framework *pytest*. Ces tests requiert Ubuntu 22.04 pour le moment.
+Consultez tests/README.md pour savoir comment les exécuter. Réussite des tests unitaires (ou une très bonne explication
+pourquoi ils échouent *temporairement*) devrait être une autre porte avant de fusionner avec la branche *developpement* principale.
+
+
 Installation locale
 -------------------
 

_sources/fr/Reference/sr3_options.7.rst.txt

@@ -1037,8 +1037,18 @@ messages de journal. Autres valeurs : on_start, on_stop, post, gather, ... etc..
 On peut débuter la valeur avec un plus (+) pour signifier un ajout au valeurs actuels.
 la valeur moins (-) signifie la soustraction des valeurs de l´ensemble actuel. 
 
+LogFormat ( default: %(asctime)s [%(levelname)s] %(name)s %(funcName)s %(message)s )
+------------------------------------------------------------------------------------
+
+L'option *LogFormat* est passée directement au mécanismes de contrôle des journalisation
+de python. Le format est documenté ici:
+
+* https://docs.python.org/fr/3/library/logging.html#logrecord-attributes
+
+
 logLevel ( défaut: info )
 -------------------------
+
 Niveau de journalisation exprimé par la journalisation de python. Les valeurs possibles sont :
 critical, error, info, warning, debug.
 

fr/Contribution/Développement.html

@@ -232,6 +232,13 @@ <h3>sr_insects<a class="headerlink" href="#sr-insects" title="Link to this headi
 Les versions 2 et 3 sont prises en charge sur la branche principale de sr_insects. Cette branche devrait être
 utilisé pour prendre en charge tout le développement dans les deux versions…</p>
 </section>
+<section id="tests-unitaires">
+<h3>Tests unitaires<a class="headerlink" href="#tests-unitaires" title="Link to this heading"></a></h3>
+<p>Le sous-répertoire tests/ contient un ensemble malheureusement incomplet mais croissant de tests unitaires
+qui utilisent le framework <em>pytest</em>. Ces tests requiert Ubuntu 22.04 pour le moment.
+Consultez tests/README.md pour savoir comment les exécuter. Réussite des tests unitaires (ou une très bonne explication
+pourquoi ils échouent <em>temporairement</em>) devrait être une autre porte avant de fusionner avec la branche <em>developpement</em> principale.</p>
+</section>
 </section>
 <section id="installation-locale">
 <h2>Installation locale<a class="headerlink" href="#installation-locale" title="Link to this heading"></a></h2>

fr/Reference/sr3_options.7.html

@@ -1039,6 +1039,14 @@ <h4>logEvents ( défaut: after_accept,after_work,on_housekeeping )<a class="head
 On peut débuter la valeur avec un plus (+) pour signifier un ajout au valeurs actuels.
 la valeur moins (-) signifie la soustraction des valeurs de l´ensemble actuel.</p>
 </section>
+<section id="logformat-default-asctime-s-levelname-s-name-s-funcname-s-message-s">
+<h4>LogFormat ( default: %(asctime)s [%(levelname)s] %(name)s %(funcName)s %(message)s )<a class="headerlink" href="#logformat-default-asctime-s-levelname-s-name-s-funcname-s-message-s" title="Link to this heading"></a></h4>
+<p>L’option <em>LogFormat</em> est passée directement au mécanismes de contrôle des journalisation
+de python. Le format est documenté ici:</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://docs.python.org/fr/3/library/logging.html#logrecord-attributes">https://docs.python.org/fr/3/library/logging.html#logrecord-attributes</a></p></li>
+</ul>
+</section>
 <section id="loglevel-defaut-info">
 <h4>logLevel ( défaut: info )<a class="headerlink" href="#loglevel-defaut-info" title="Link to this heading"></a></h4>
 <p>Niveau de journalisation exprimé par la journalisation de python. Les valeurs possibles sont :