Merge pull request #20 from BNLNPPS/maxim-refactor

Updated documentation

Author: buddhasystem

README.md

@@ -4,8 +4,8 @@ Common libraries and utilities for the swf-testbed ePIC streaming workflow testb
 
 ## Overview
 
-This library provides shared functionality for SWF agents, including logging utilities and REST API integration for the swf-monitor service.
-Also included are utility classes wrapping MQ and Rucio communications.
+This library provides shared functionality for SWF agents, including logging utilities and REST API
+integration for the swf-monitor service. Also included are utility classes wrapping MQ and Rucio communications.
 
 ## Installation
 
@@ -85,4 +85,10 @@ When the monitor service is available, logs are sent to the database via REST AP
 
 ### Logging Utils (`swf_common_lib.logging_utils`)
 
-Traditional logging utilities for PostgreSQL database integration.
+Traditional logging utilities for PostgreSQL database integration.
+
+## MQ and Rucio Utility packages
+
+The *mq_comms* and *rucio_comms* packages provide convenient encapsulation of interactions
+with the ActiveMQ and Rucio systems, respectively. Each folder contains it's own README file
+with more details.

mq_comms/README.md

@@ -34,3 +34,8 @@ a set defined as follows:
         'stf_gen', 'data_ready'
     }
 ```
+
+## Subscription
+
+For the _Receiver_ class, it's important to create a unique subscription for
+reliable delivery of messages. This is done via the argument *client_id*.

rucio_comms/README.md

@@ -0,0 +1,36 @@
+# Rucio comms
+
+The rucio_comms package is a Python library that provides a high-level interface for interacting with Rucio. Here are its key components and features:
+
+1. Core Classes:
+- `FileManager`: Manages Rucio file operations
+  - Registers file replicas with existing PFNs
+  - Associates files with datasets
+  - Handles batch operations for multiple files
+  - Tracks registered files
+
+- `DatasetManager`: Handles Rucio dataset operations
+  - Creates and manages datasets
+  - Sets metadata, lifetime, and status
+  - Follows PanDA-style dataset management patterns
+
+- `FileInfo`: Represents a file with its metadata
+  - Handles logical/physical file names (LFN/PFN)
+  - Manages checksums, size, and GUIDs
+  - Validates file attributes
+
+2. Utility Modules:
+- `RucioUtils`: Helper functions for Rucio operations
+  - Scope extraction from dataset names
+  - GUID and VUID generation
+  - PFN parsing and formatting
+
+- `ValidationUtils`: Input validation
+  - Validates dataset names, scopes, LFNs, PFNs
+  - Checks checksums and file sizes
+  - Enforces format rules
+
+- `MetadataUtils`: Metadata handling
+  - Creates standardized file metadata
+  - Manages dataset metadata
+  - Follows PanDA/Rucio metadata conventions