doc: use plantweb as fallback of sphinx-ditaa

RTD does not support installing system packages, the only ways to install dependencies are setuptools and pip. while ditaa is a tool written in Java. so we need to find a native python tool allowing us to render ditaa images. plantweb is able to the web service for rendering the ditaa diagram. so let's use it as a fallback if "ditaa" is not around. also start a new line after the directive, otherwise planweb server will return 500 at seeing the diagram. Signed-off-by: Kefu Chai <[email protected]>
biohazd · Apr 10, 2020 · 0cb56e0 · 0cb56e0
1 parent 847e4ef
commit 0cb56e0
Show file tree

Hide file tree

Showing 36 changed files with 183 additions and 67 deletions.
diff --git a/.readthedocs.yml b/.readthedocs.yml
@@ -10,6 +10,7 @@ python:
   version: 3
   install:
     - requirements: admin/doc-requirements.txt
+    - requirements: admin/doc-read-the-docs.txt
 sphinx:
   builder: dirhtml
   configuration: doc/conf.py
diff --git a/admin/doc-read-the-docs.txt b/admin/doc-read-the-docs.txt
@@ -0,0 +1 @@
+plantweb
diff --git a/doc/architecture.rst b/doc/architecture.rst
@@ -27,7 +27,9 @@ A Ceph Storage Cluster consists of two types of daemons:
 - :term:`Ceph Monitor`
 - :term:`Ceph OSD Daemon`
 
-.. ditaa::  +---------------+ +---------------+
+.. ditaa::
+
+            +---------------+ +---------------+
             |      OSDs     | |    Monitors   |
             +---------------+ +---------------+
 
@@ -56,7 +58,9 @@ comes through a :term:`Ceph Block Device`, :term:`Ceph Object Storage`, the
 file in a filesystem, which is stored on an :term:`Object Storage Device`. Ceph
 OSD Daemons handle the read/write operations on the storage disks.
 
-.. ditaa:: /-----\       +-----+       +-----+
+.. ditaa::
+
+           /-----\       +-----+       +-----+
            | obj |------>| {d} |------>| {s} |
            \-----/       +-----+       +-----+
 
@@ -70,7 +74,9 @@ attributes such as the file owner, created date, last modified date, and so
 forth.
 
 
-.. ditaa:: /------+------------------------------+----------------\
+.. ditaa::
+
+           /------+------------------------------+----------------\
            | ID   | Binary Data                  | Metadata       |
            +------+------------------------------+----------------+
            | 1234 | 0101010101010100110101010010 | name1 = value1 | 
@@ -229,7 +235,9 @@ the client and the monitor share a secret key.
 .. note:: The ``client.admin`` user must provide the user ID and 
    secret key to the user in a secure manner. 
 
-.. ditaa:: +---------+     +---------+
+.. ditaa::
+
+           +---------+     +---------+
            | Client  |     | Monitor |
            +---------+     +---------+
                 |  request to   |
@@ -252,7 +260,9 @@ user's secret key and transmits it back to the client. The client decrypts the
 ticket and uses it to sign requests to OSDs and metadata servers throughout the
 cluster.
 
-.. ditaa:: +---------+     +---------+
+.. ditaa::
+
+           +---------+     +---------+
            | Client  |     | Monitor |
            +---------+     +---------+
                 |  authenticate |
@@ -283,7 +293,9 @@ machine and the Ceph servers. Each message sent between a client and server,
 subsequent to the initial authentication, is signed using a ticket that the
 monitors, OSDs and metadata servers can verify with their shared secret.
 
-.. ditaa:: +---------+     +---------+     +-------+     +-------+
+.. ditaa::
+
+           +---------+     +---------+     +-------+     +-------+
            |  Client |     | Monitor |     |  MDS  |     |  OSD  |
            +---------+     +---------+     +-------+     +-------+
                 |  request to   |              |             |
@@ -393,7 +405,8 @@ ability to leverage this computing power leads to several major benefits:
    and tertiary OSDs (as many OSDs as additional replicas), and responds to the
    client once it has confirmed the object was stored successfully.
 
-.. ditaa:: 
+.. ditaa::
+
              +----------+
              |  Client  |
              |          |
@@ -443,7 +456,8 @@ Ceph Clients retrieve a `Cluster Map`_ from a Ceph Monitor, and write objects to
 pools. The pool's ``size`` or number of replicas, the CRUSH rule and the
 number of placement groups determine how Ceph will place the data.
 
-.. ditaa:: 
+.. ditaa::
+
             +--------+  Retrieves  +---------------+
             | Client |------------>|  Cluster Map  |
             +--------+             +---------------+
@@ -488,7 +502,8 @@ rebalance dynamically when new Ceph OSD Daemons and the underlying OSD devices
 come online. The following diagram depicts how CRUSH maps objects to placement
 groups, and placement groups to OSDs.
 
-.. ditaa:: 
+.. ditaa::
+
            /-----\  /-----\  /-----\  /-----\  /-----\
            | obj |  | obj |  | obj |  | obj |  | obj |
            \-----/  \-----/  \-----/  \-----/  \-----/
@@ -614,7 +629,8 @@ and each OSD gets some added capacity, so there are no load spikes on the
 new OSD after rebalancing is complete.
 
 
-.. ditaa:: 
+.. ditaa::
+
            +--------+     +--------+
    Before  |  OSD 1 |     |  OSD 2 |
            +--------+     +--------+
@@ -685,6 +701,7 @@ name. Chunk 1 contains ``ABC`` and is stored on **OSD5** while chunk 4 contains
 
 
 .. ditaa::
+
                             +-------------------+
                        name |       NYAN        |
                             +-------------------+
@@ -739,6 +756,7 @@ three chunks are read: **OSD2** was the slowest and its chunk was not taken into
 account.
 
 .. ditaa::
+
 	                         +-------------------+
 	                    name |       NYAN        |
 	                         +-------------------+
@@ -804,6 +822,7 @@ version 1).
 
 
 .. ditaa::
+
      Primary OSD
 
    +-------------+
@@ -934,6 +953,7 @@ object can be removed: ``D1v1`` on **OSD 1**, ``D2v1`` on **OSD 2** and ``C1v1``
 on **OSD 3**.
 
 .. ditaa::
+
      Primary OSD
 
    +-------------+
@@ -972,6 +992,7 @@ to be available on all OSDs in the previous acting set ) is ``1,1`` and that
 will be the head of the new authoritative log.
 
 .. ditaa::
+
    +-------------+
    |    OSD 1    |
    |   (down)    |
@@ -1017,6 +1038,7 @@ the erasure coding library during scrubbing and stored on the new primary
 
 
 .. ditaa::
+
      Primary OSD
 
    +-------------+
@@ -1068,7 +1090,8 @@ tier. So the cache tier and the backing storage tier are completely transparent
 to Ceph clients.
 
 
-.. ditaa:: 
+.. ditaa::
+
            +-------------+
            | Ceph Client |
            +------+------+
@@ -1150,7 +1173,8 @@ Cluster. Ceph packages this functionality into the ``librados`` library so that
 you can create your own custom Ceph Clients. The following diagram depicts the
 basic architecture.
 
-.. ditaa::  
+.. ditaa::
+
             +---------------------------------+
             |  Ceph Storage Cluster Protocol  |
             |           (librados)            |
@@ -1193,7 +1217,9 @@ notification. This enables a client to use any object as a
 synchronization/communication channel.
 
 
-.. ditaa:: +----------+     +----------+     +----------+     +---------------+
+.. ditaa::
+
+           +----------+     +----------+     +----------+     +---------------+
            | Client 1 |     | Client 2 |     | Client 3 |     | OSD:Object ID |
            +----------+     +----------+     +----------+     +---------------+
                  |                |                |                  |
@@ -1269,7 +1295,8 @@ take maximum advantage of Ceph's ability to distribute data across placement
 groups, and consequently doesn't improve performance very much. The following
 diagram depicts the simplest form of striping:
 
-.. ditaa::              
+.. ditaa::
+
                         +---------------+
                         |  Client Data  |
                         |     Format    |
@@ -1327,7 +1354,8 @@ set (``object set 2`` in the following diagram), and begins writing to the first
 stripe (``stripe unit 16``) in the first object in the new object set (``object
 4`` in the diagram below).
 
-.. ditaa::                 
+.. ditaa::
+
                           +---------------+
                           |  Client Data  |
                           |     Format    |
@@ -1443,6 +1471,7 @@ and high availability. The following diagram depicts the high-level
 architecture. 
 
 .. ditaa::
+
             +--------------+  +----------------+  +-------------+
             | Block Device |  | Object Storage |  |   CephFS    |
             +--------------+  +----------------+  +-------------+            
@@ -1527,6 +1556,7 @@ Cluster. Ceph Clients mount a CephFS filesystem as a kernel object or as
 a Filesystem in User Space (FUSE).
 
 .. ditaa::
+
             +-----------------------+  +------------------------+
             | CephFS Kernel Object  |  |      CephFS FUSE       |
             +-----------------------+  +------------------------+            

diff --git a/doc/cephfs/cephfs-io-path.rst b/doc/cephfs/cephfs-io-path.rst
@@ -19,6 +19,7 @@ client cache.
 
 
 .. ditaa::
+
             +---------------------+
             |     Application     |
             +---------------------+

diff --git a/doc/conf.py b/doc/conf.py
@@ -1,3 +1,4 @@
+import shutil
 import sys
 import os
 
@@ -53,12 +54,19 @@
     'sphinx_autodoc_typehints',
     'sphinx.ext.graphviz',
     'sphinx.ext.todo',
-    'sphinxcontrib.ditaa',
     'breathe',
     'edit_on_github',
     'ceph_releases',
     ]
-ditaa = 'ditaa'
+ditaa = shutil.which("ditaa")
+if ditaa is not None:
+    extensions += ['sphinxcontrib.ditaa']
+else:
+    extensions += ['plantweb.directive']
+    plantweb_defaults = {
+        'engine': 'ditaa'
+    }
+
 todo_include_todos = True
 
 top_level = os.path.dirname(
@@ -87,6 +95,10 @@
 # handles edit-on-github and old version warning display
 def setup(app):
     app.add_javascript('js/ceph.js')
+    if ditaa is None:
+        # add "ditaa" as an alias of "diagram"
+        from plantweb.directive import DiagramDirective
+        app.add_directive('ditaa', DiagramDirective)
 
 # mocking ceph_module offered by ceph-mgr. `ceph_module` is required by
 # mgr.mgr_module

diff --git a/doc/dev/deduplication.rst b/doc/dev/deduplication.rst
@@ -50,7 +50,8 @@ More details in https://ieeexplore.ieee.org/document/8416369
 Design
 ======
 
-.. ditaa:: 
+.. ditaa::
+
            +-------------+
            | Ceph Client |
            +------+------+

diff --git a/doc/dev/msgr2.rst b/doc/dev/msgr2.rst
@@ -65,7 +65,9 @@ If the remote party advertises required features we don't support, we
 can disconnect.
 
 
-.. ditaa:: +---------+        +--------+
+.. ditaa::
+
+           +---------+        +--------+
            | Client  |        | Server |
            +---------+        +--------+
                 | send banner     |
@@ -159,7 +161,9 @@ Authentication
 Example of authentication phase interaction when the client uses an
 allowed authentication method:
 
-.. ditaa:: +---------+        +--------+
+.. ditaa::
+
+           +---------+        +--------+
            | Client  |        | Server |
            +---------+        +--------+
                 | auth request    |
@@ -176,7 +180,9 @@ allowed authentication method:
 Example of authentication phase interaction when the client uses a forbidden
 authentication method as the first attempt:
 
-.. ditaa:: +---------+        +--------+
+.. ditaa::
+
+           +---------+        +--------+
            | Client  |        | Server |
            +---------+        +--------+
                 | auth request    |
@@ -333,7 +339,9 @@ Example of failure scenarios:
 
 * First client's client_ident message is lost, and then client reconnects.
 
-.. ditaa:: +---------+           +--------+
+.. ditaa::
+
+           +---------+           +--------+
            | Client  |           | Server |
            +---------+           +--------+
                 |                     |
@@ -351,7 +359,9 @@ Example of failure scenarios:
 
 * Server's server_ident message is lost, and then client reconnects.
 
-.. ditaa:: +---------+           +--------+
+.. ditaa::
+
+           +---------+           +--------+
            | Client  |           | Server |
            +---------+           +--------+
                 |                     |
@@ -372,7 +382,9 @@ Example of failure scenarios:
 
 * Server's server_ident message is lost, and then server reconnects.
 
-.. ditaa:: +---------+           +--------+
+.. ditaa::
+
+           +---------+           +--------+
            | Client  |           | Server |
            +---------+           +--------+
                 |                     |
@@ -396,7 +408,9 @@ Example of failure scenarios:
 
 * Connection failure after session is established, and then client reconnects.
 
-.. ditaa:: +---------+           +--------+
+.. ditaa::
+
+           +---------+           +--------+
            | Client  |           | Server |
            +---------+           +--------+
                 |                     |
@@ -414,7 +428,9 @@ Example of failure scenarios:
 * Connection failure after session is established because server reset,
   and then client reconnects.
 
-.. ditaa:: +---------+           +--------+
+.. ditaa::
+
+           +---------+           +--------+
            | Client  |           | Server |
            +---------+           +--------+
                 |                     |
@@ -440,7 +456,9 @@ of the connection.
 * Connection failure after session is established because client reset,
   and then client reconnects.
 
-.. ditaa:: +---------+           +--------+
+.. ditaa::
+
+           +---------+           +--------+
            | Client  |           | Server |
            +---------+           +--------+
                 |                     |
@@ -507,7 +525,9 @@ Example of protocol interaction (WIP)
 _____________________________________
 
 
-.. ditaa:: +---------+        +--------+
+.. ditaa::
+
+           +---------+        +--------+
            | Client  |        | Server |
            +---------+        +--------+
                 | send banner       |
-Original file line number
+Diff line change
@@ Expand Up / @@ -19,6 +19,7 @@ client cache. @@
     .. ditaa::
                 +---------------------+
                 |     Application     |
                 +---------------------+
@@ Expand Down @@