Some more docs work

docs/conf.py

@@ -18,7 +18,7 @@
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
 project = 'The Deed Machine'
-copyright = f'{date.today().year}, Mapping Prejudice'
+copyright = f'{date.today().year} Regents of the University of Minnesota'
 author = 'Mapping Prejudice'
 release = '1.0 dev1'
 

docs/index.rst

@@ -4,16 +4,35 @@
    contain the root `toctree` directive.
 
 The Deed Machine
-============================================
+================
+
+.. image:: https://s3.us-east-2.amazonaws.com/static.mappingprejudice.com/deed-machine/MappingPrejudice_C_600.png
+   :width: 200
+   :align: right
+   :alt: Mapping Prejudice logo
 
 The Deed Machine is a multi-language set of tools that use OCR and crowdsourced transcription to identify racially restrictive covenant language, then map the results.
 
+Racial covenants are clauses that were inserted into property deeds to prevent people who are not white from buying or occupying land. As of June 2024, Mapping Prejudice volunteers have helped to map `more than 52,000 covenanted properties <https://github.com/umnlibraries/mp-us-racial-covenants>`_ across 3 states, with more on the way soon.
+
 The Deed Machine was created at Mapping Prejudice at the University of Minnesota Libraries. Current collaborators include Michael Corey, Suleman Diwan, Justin Schell, and the University of Minnesota Libraries IT staff.
 
 .. image:: https://s3.us-east-2.amazonaws.com/static.mappingprejudice.com/deed-machine/Draft%20-%20Updated%20Workflow_alpha.png
   :width: 800
   :alt: A diagram of the components of the Deed Machine. To the left, an initial processing stage using AWS Step Functions is used to output a series of S3 files, which are ingested into a Django project in the center of the diagram.
 
+.. toctree::
+   :maxdepth: 2
+   :caption: The Deed Machine
+
+   modules/funding.rst
+   modules/requirements.rst
+   modules/components.rst
+   modules/installation.rst
+   modules/development.rst
+   modules/support.rst
+   modules/license.rst
+
 .. toctree::
    :maxdepth: 2
    :caption: Common workflows

docs/modules/apps-deed-models.rst

@@ -1,4 +1,4 @@
 apps.deed.models.py
 ===================
 .. automodule:: apps.deed.models
-   :members:
+   :members:

docs/modules/components.rst

@@ -0,0 +1,20 @@
+Components
+==========
+
+Standalone deed uploader
+------------------------
+
+Often deed images are stored on a local machine or network drive, and it's not feasible or efficient to move them. This standalone uploader is designed to avoid the user having to do a full install on this computer.
+
+- `mp-upload-deed-images-standalone <https://github.com/UMNLibraries/mp-upload-deed-images-standalone>`_
+
+Lambda functions used for OCR step machine
+------------------------------------------
+
+The individual lambda functions that make up the OCR, term search and web image optimization processes are in separate repositories:
+
+- `mp-covenants-split-pages <https://github.com/UMNLibraries/mp-covenants-split-pages>`_
+- `mp-covenants-ocr-page <https://github.com/UMNLibraries/mp-covenants-ocr-page>`_
+- `mp-covenants-term-search-basic <https://github.com/UMNLibraries/mp-covenants-term-search-basic>`_
+- `mp-covenants-resize-image <https://github.com/UMNLibraries/mp-covenants-resize-image>`_
+- `mp-covenants-fake-ocr <https://github.com/UMNLibraries/mp-covenants-fake-ocr>`_

docs/modules/development.rst

@@ -0,0 +1,4 @@
+Development
+===========
+
+If you would like to help with Deed Machine software development, including helping with additional documentation, please contact `Michael Corey <https://github.com/mikejcorey>`_ at `Mapping Prejudice <https://mappingprejudice.umn.edu/>`_.

docs/modules/downloading-new-results.rst

@@ -65,3 +65,4 @@ Downloading new batches of Zooniverse results
     python manage.py dump_covenants_shapefile --workflow "WI Milwaukee County"
     python manage.py dump_covenants_geojson --workflow "WI Milwaukee County"
     python manage.py dump_covenants_csv --workflow "WI Milwaukee County"
+

docs/modules/funding.rst

@@ -0,0 +1,4 @@
+Funding
+=======
+
+TK TK TK

docs/modules/installation.rst

@@ -0,0 +1,52 @@
+Installation
+============
+
+Django installation process
+---------------------------
+
+1. Create a PostGIS-enabled database for the project
+The psql command will vary slightly with different OSes. For Mac:
+
+.. code-block:: bash
+
+    psql -d postgres
+
+    CREATE DATABASE racial_covenants_processor;
+    CREATE USER racial_covenants_processor with password 'racial_covenants_processor';
+    GRANT ALL PRIVILEGES ON DATABASE racial_covenants_processor to racial_covenants_processor;
+    ALTER DATABASE racial_covenants_processor OWNER TO racial_covenants_processor;
+    \q
+    psql -d racial_covenants_processor
+    CREATE EXTENSION postgis;
+
+
+2. Install Python environment
+
+.. code-block:: bash
+
+    pipenv install
+
+3. Create a Postgresql service to connect between Django and the DB called ``.pg_service.conf``
+
+.. code-block::
+
+    [deeds_service]
+    host=localhost
+    user=racial_covenants_processor
+    dbname=racial_covenants_processor
+    port=5432
+
+
+4. Sync Django with your database
+
+.. code-block:: bash
+
+    pipenv shell
+    python manage.py migrate
+
+
+5. To be able to view the admin pages, create a superuser
+
+.. code-block:: bash
+
+    python manage.py createsuperuser

docs/modules/license.rst

@@ -0,0 +1,39 @@
+.. |date| date::
+
+License
+=======
+
+Deed Machine software
+---------------------
+
+MIT License
+
+Copyright © 2024 Regents of the University of Minnesota
+
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+
+Racial covenants data
+---------------------
+
+Covenants data produced by the Deed Machine and published by Mapping Prejudice is released under a CCO license.
+
+See `U.S. racial covenants data <https://github.com/UMNLibraries/mp-us-racial-covenants>`_ repository.
+

docs/modules/manual-data-cleaning.rst

@@ -21,4 +21,4 @@ Once all of the steps used to process new results have been run, many covenants
 
 9. Choose a "match type" value to indicate how this parcel was matched (or how it will need to be matched in the future).
 
-10. If you need to map lots across more than one block, add ExtraParcelCandidate objects for each additional block or lot range. ONLY ONE ManualCorrection OBJECT should be added per ZooniverseSubject.
+10. If you need to map lots across more than one block, add ExtraParcelCandidate objects for each additional block or lot range. ONLY ONE ManualCorrection OBJECT should be added per ZooniverseSubject.

docs/modules/requirements.rst

@@ -0,0 +1,34 @@
+Requirements
+============
+
+The Deed Machine uses a combination of Amazon Web Services (AWS), Django, and standalone Python components.
+
+AWS services used
+-----------------
+
+- S3
+- Lambda
+- Step Functions
+- Textract
+- RDS
+- ECL/ECS
+- EC2
+- AWS sam (for lambda deployment)
+
+Django software requirements
+----------------------------
+
+In addition to the Python requirements listed in the Pipfile, to run the Deed Machine's Django components locally, you will need:
+
+- geos and gdal
+- proj
+- geopandas (Python, but can require persnickety configuratino)
+- pyenv 
+- pipenv
+- PostgreSQL/PostGIS
+- libmagic (mostly to silence panoptes/zooniverse warnings)
+
+In order to deploy Django components to production, you will additionally need:
+
+- Docker
+- AWS CLI

docs/modules/support.rst

@@ -0,0 +1,4 @@
+Support
+=======
+
+Do to the early phase of work and high demand, at this time, we are only able to provided technical support on a limited basis as time allows. Please contact `Michael Corey <https://github.com/mikejcorey>`_ at `Mapping Prejudice <https://mappingprejudice.umn.edu/>`_.