Harmonize Envoy CLI and Configuration Files with Director Component f…

…or Consistency (#1284)

* Clean envoy CLI & config

Signed-off-by: refai06 <[email protected]>

* Cleanup federatedruntime testcase-config

Signed-off-by: refai06 <[email protected]>

* Update files

Signed-off-by: refai06 <[email protected]>

* Envoy execption-handling

Signed-off-by: refai06 <[email protected]>

* Update error messages

Signed-off-by: refai06 <[email protected]>

* Documentation update for recent change

Signed-off-by: refai06 <[email protected]>

* Fix code format

Signed-off-by: refai06 <[email protected]>

* Document Update

Signed-off-by: refai06 <[email protected]>

---------

Signed-off-by: refai06 <[email protected]>

docs/about/features_index/workflowinterface.rst

@@ -342,37 +342,43 @@ An example configuration file `director_config.yaml` is shown below:
 
 **Envoy: Participating nodes in the Federation**
 
-The `fx envoy start` command is used to start the Envoy. You can run it with or without TLS, depending on your setup.
+The :code:`fx envoy start` command is used to start the Envoy. You can run it with or without TLS, depending on your setup.
 
 **With TLS:**
 Use the following command:
 
 .. code-block:: shell
 
-    $ fx envoy start -n <envoy_name> -ec <path_to_envoy_config_yaml_file> -dh <director_host> -dp <director_port> -rc <root_certificate_path> -pk <private_key_path> -oc <api_certificate_path>
+    $ fx envoy start -n <envoy_name> -c <path_to_envoy_config_yaml_file> -rc <root_certificate_path> -pk <private_key_path> -oc <api_certificate_path>
 
 **Without TLS:**
 Use the following command:
 
 .. code-block:: shell
 
-    $ fx envoy start -n <envoy_name> --disable-tls -ec <path_to_envoy_config_yaml_file>
+    $ fx envoy start -n <envoy_name> --disable-tls -c <path_to_envoy_config_yaml_file>
 
 **Explanation of Command Options**
 
 - `-n <envoy_name>`: Specifies the name of the Envoy.
-- `-ec <path_to_envoy_config_yaml_file>`: Path to the Envoy's configuration file.
-- `-dh <director_host>`: Hostname or IP address of the Director.
-- `-dp <director_port>`: Port on which the Director is running.
+- `-c <path_to_envoy_config_yaml_file>`: Path to the Envoy's configuration file.
 - `-rc <root_certificate_path>`: Path to the root certificate (used with TLS).
 - `-pk <private_key_path>`: Path to the private key file (used with TLS).
 - `-oc <api_certificate_path>`: Path to the API certificate file (used with TLS).
 - `--disable-tls`: Disables TLS encryption.
 
-The Envoy configuration file includes details about the private attributes. An example configuration file :code:`envoy_config.yaml` for :code:`envoy_one` is shown below:
+The Envoy configuration file includes details of director_host, director_port and private attributes. An example configuration file :code:`envoy_config.yaml` for :code:`settings` and :code:`envoy_one` is shown below:
+
+- Hostname (`director_host`)
+- Port (`director_port`)
+- Private attributes for envoy_one
 
 .. code-block:: yaml
 
+   settings:
+       director_host: localhost
+       director_port: 50050
+
    envoy_one:
        private_attributes: private_attributes.envoy_one_attrs
 

openfl-tutorials/experimental/workflow/FederatedRuntime/101_MNIST/Portland/Portland_config.yaml

@@ -1,2 +1,6 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 Portland:
   private_attributes: private_attributes.portland_attrs

openfl-tutorials/experimental/workflow/FederatedRuntime/101_MNIST/Portland/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

openfl-tutorials/experimental/workflow/FederatedRuntime/101_MNIST/Seattle/Seattle_config.yaml

@@ -1,2 +1,6 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 Seattle:
   private_attributes: private_attributes.seattle_attrs

openfl-tutorials/experimental/workflow/FederatedRuntime/101_MNIST/Seattle/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

openfl-tutorials/experimental/workflow/FederatedRuntime/301_MNIST_Watermarking/Bangalore/Bangalore_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 Bangalore:
   callable_func:
     settings:

openfl-tutorials/experimental/workflow/FederatedRuntime/301_MNIST_Watermarking/Bangalore/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

openfl-tutorials/experimental/workflow/FederatedRuntime/301_MNIST_Watermarking/Chandler/Chandler_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 Chandler:
   callable_func:
     settings:

openfl-tutorials/experimental/workflow/FederatedRuntime/301_MNIST_Watermarking/Chandler/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

openfl/experimental/workflow/interface/cli/envoy.py

@@ -12,7 +12,7 @@
 from dynaconf import Validator
 
 from openfl.experimental.workflow.component.envoy import Envoy
-from openfl.utilities import click_types, merge_configs
+from openfl.utilities import is_fqdn, merge_configs
 from openfl.utilities.path_check import is_directory_traversal
 
 logger = logging.getLogger(__name__)
@@ -32,32 +32,18 @@ def envoy(context):
 @envoy.command(name="start")
 @option("-n", "--envoy_name", required=True, help="Current shard name")
 @option(
-    "-dh",
-    "--director-host",
-    required=True,
-    help="The FQDN of the federation director",
-    type=click_types.FQDN,
-)
-@option(
-    "-dp",
-    "--director-port",
-    required=True,
-    help="The federation director port",
-    type=click.IntRange(1, 65535),
+    "-c",
+    "--envoy-config-path",
+    default="envoy_config.yaml",
+    help="The envoy config path",
+    type=ClickPath(exists=True),
 )
 @option(
     "--tls/--disable-tls",
     default=True,
     is_flag=True,
     help="Use TLS or not (By default TLS is enabled)",
 )
-@option(
-    "-ec",
-    "--envoy-config-path",
-    default="envoy_config.yaml",
-    help="The envoy config path",
-    type=ClickPath(exists=True),
-)
 @option(
     "-rc",
     "--root-cert-path",
@@ -84,8 +70,6 @@ def envoy(context):
 )
 def start_(
     envoy_name,
-    director_host,
-    director_port,
     tls,
     envoy_config_path,
     root_certificate,
@@ -96,8 +80,6 @@ def start_(
 
     Args:
         envoy_name (str): Name of the Envoy.
-        director_host (str): The FQDN of the federation director.
-        director_port (int): The federation director port.
         tls (bool): Use TLS or not.
         envoy_config_path (str): The envoy config path.
         root_certificate (str): Path to a root CA cert.
@@ -118,6 +100,31 @@ def start_(
             "certificate": certificate,
         },
         validators=[
+            Validator(
+                "settings",
+                must_exist=True,
+                messages={"must_exist_true": "Missing 'settings' in Envoy Configuration file."},
+            ),
+            Validator(
+                "settings.director_host",
+                must_exist=True,
+                condition=lambda x: bool(x) and is_fqdn(x),
+                messages={
+                    "must_exist_true": "Missing 'director_host' in Envoy Configuration file",
+                    "condition": "Invalid 'director_host' in Envoy Configuration file."
+                    "Must be a valid FQDN",
+                },
+            ),
+            Validator(
+                "settings.director_port",
+                must_exist=True,
+                condition=lambda value: isinstance(value, int) and 1024 <= value <= 65535,
+                messages={
+                    "must_exist_true": "Missing 'director_port' in Envoy Configuration file",
+                    "condition": "Invalid 'director_port' in Envoy Configuration file"
+                    "Must be an integer between 1024 & 65535",
+                },
+            ),
             Validator("params.install_requirements", default=True),
         ],
     )
@@ -138,8 +145,8 @@ def start_(
 
     envoy = Envoy(
         envoy_name=envoy_name,
-        director_host=director_host,
-        director_port=director_port,
+        director_host=config.settings.director_host,
+        director_port=config.settings.director_port,
         envoy_config=Path(envoy_config_path).absolute(),
         root_certificate=config.root_certificate,
         private_key=config.private_key,

tests/github/experimental/workflow/FederatedRuntime/testcase_datastore_cli/envoy_one/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_one:
   callable_func:
     settings:

tests/github/experimental/workflow/FederatedRuntime/testcase_datastore_cli/envoy_one/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_datastore_cli/envoy_two/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_two:
   callable_func:
     settings:

tests/github/experimental/workflow/FederatedRuntime/testcase_datastore_cli/envoy_two/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_include_exclude/envoy_one/envoy_config.yaml

@@ -1,2 +1,4 @@
-
+settings:
+  director_host: localhost
+  director_port: 50050
 

tests/github/experimental/workflow/FederatedRuntime/testcase_include_exclude/envoy_one/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_include_exclude/envoy_two/envoy_config.yaml

@@ -0,0 +1,4 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+

tests/github/experimental/workflow/FederatedRuntime/testcase_include_exclude/envoy_two/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_internalloop/envoy_one/envoy_config.yaml

@@ -0,0 +1,3 @@
+settings:
+  director_host: localhost
+  director_port: 50050

tests/github/experimental/workflow/FederatedRuntime/testcase_internalloop/envoy_one/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_internalloop/envoy_two/envoy_config.yaml

@@ -0,0 +1,3 @@
+settings:
+  director_host: localhost
+  director_port: 50050

tests/github/experimental/workflow/FederatedRuntime/testcase_internalloop/envoy_two/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes/envoy_one/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_one:
   callable_func:
     settings:

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes/envoy_one/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes/envoy_two/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_two:
   callable_func:
     settings:

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes/envoy_two/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_with_both_options/envoy_one/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_one:
   callable_func:
     settings:

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_with_both_options/envoy_one/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_with_both_options/envoy_two/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_two:
   callable_func:
     settings:

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_with_both_options/envoy_two/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_without_callable/envoy_one/envoy_config.yaml

@@ -1,2 +1,6 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_one:
   private_attributes: private_attributes.collaborator_private_attrs.collaborator_private_attributes

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_without_callable/envoy_one/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_without_callable/envoy_two/envoy_config.yaml

@@ -1,2 +1,6 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_two:
   private_attributes: private_attributes.collaborator_private_attrs.collaborator_private_attributes

tests/github/experimental/workflow/FederatedRuntime/testcase_private_attributes_initialization_without_callable/envoy_two/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_reference/envoy_one/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_one:
   callable_func:
     settings:

tests/github/experimental/workflow/FederatedRuntime/testcase_reference/envoy_one/start_envoy.sh

@@ -3,4 +3,4 @@ set -e
 ENVOY_NAME=$1
 ENVOY_CONF=$2
 
-fx envoy start -n "$ENVOY_NAME" --disable-tls --envoy-config-path "$ENVOY_CONF" -dh localhost -dp 50050
+fx envoy start -n "$ENVOY_NAME" --disable-tls -c "$ENVOY_CONF"

tests/github/experimental/workflow/FederatedRuntime/testcase_reference/envoy_two/envoy_config.yaml

@@ -1,3 +1,7 @@
+settings:
+  director_host: localhost
+  director_port: 50050
+
 envoy_two:
   callable_func:
     settings: