Skip to content

Commit

Permalink
Updating Database Model (Quality Model, Adapatation Plan) (#7)
Browse files Browse the repository at this point in the history
* Data models for Metrics.
* Fixing the type of the primary key of the Quality Model
* JSON files with the formats of the messages
* Updating diagrams of the data models (metrics)
  • Loading branch information
joseadp authored and nmsa committed Oct 26, 2018
1 parent dc4aa3f commit 925fd92
Show file tree
Hide file tree
Showing 28 changed files with 74 additions and 4 deletions.
Binary file modified architecture/TMA-Framework.asta
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed architecture/diagrams/TMA-E/TMA-E_Actuation.jpg
Binary file not shown.
Binary file added architecture/diagrams/TMA-E/TMA-E_Actuation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed architecture/diagrams/TMA-E/TMA-E_Registration.jpg
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed architecture/diagrams/TMA-M/TMA-M_sequence.jpg
Binary file not shown.
Binary file added architecture/diagrams/TMA-M/TMA-M_sequence.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 27 additions & 0 deletions documents/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Trustworthiness Monitoring & Assessment Framework

The communication between the components from the `MAPE-K` is done mainly by the `FaultTolerantQueue`, according to the [figure below](../architecture/diagrams/TMA-Platform_Detailed_Architecture.jpg).

*![High level architecture of TMA Framework](../architecture/diagrams/TMA-Platform_Detailed_Architecture.jpg) Architecture and interfaces of the trustworthiness platform.*

The [`TMA_Analyze`](#tma_analyze) component is responsible for continuously listening to the `TMA_Knowledge` microservice, calculate the trustworthiness scores and, store them on the database. An item is added to the `planning` queue from the `FaultTolerantQueue` in order to the [`TMA_Planning`](#tma_planning) starts checking the rules. The format of the message is a [JSON](interface/atmosphere_tma-p_schema.json) according to the image below:

*![JSON Format read by TMA_Planning](interface/atmosphere_tma-p_schema.png) Format of the JSON message to be read by the TMA_Planning component*

The `metricId` indicates that a new value for the corresponding metric was calculated and stored in the database, in the moment stored on `timestamp`.

The [`TMA_Planning`](#tma_planning) microservice is responsible for executing the verification rules that will produce a set of adaptations that will be consolidated in an "adaptation plan" to achieve the required goals, or to recover the desired levels of trustworthiness. This is done by continuously polling the messages from the `planning` queue. If an adaptation is needed, an "adaptation plan" is stored in the database with the set of actions.

*![Conceptual Data Model used by the Planning component](../architecture/diagrams/TMA-K_DataModel/TMAF-K_ConceptualDataModel_Planning.jpg) Conceptual Data Model used by the Planning component.*

An adaptation plan is stored in the `Plan` table, and it is always related to the `MetricData` that dispatched the adapation. The set of actions and the order of execution is stored in the `ActionPlan` table, and it is related to an `Action`, which will be performed by the correspondent `Actuator`. In case the actions need parameters, their values are stored in the `ConfigurationData` table.

The communication of [`TMA_Planning`](#tma_planning) is done by the `execute` queue, which will receive a [JSON](interface/atmosphere_tma-e_schema.json) message with the plan identifier.

*![JSON Format read by TMA_Execute](interface/atmosphere_tma-e_schema.png) Format of the JSON message to be read by the TMA_Execute component*

Finally, the [`TMA_Execute`](#tma_execute) microservice runs the adaptation plan using the deployed actuators on the target layers. This component is continuously polling the messages from the `execute` queue. When a plan is added, the list of actions is obtained from the [`TMA_Knowledge`](#tma_knowledge), with the order of execution and the parameters values.

(nmsa: CHECK IF THIS CONTENT SHOULD REPLACE THE SECTION FROM THE MAIN PAGE)

(REVIEW THE LINKS TO THE SUB-MODULES)
18 changes: 18 additions & 0 deletions documents/interface/atmosphere_tma-e_schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"title": "tma-e_schema_0_1",
"$id": "http://atmosphere-eubrazil.eu/tma-e_schema_v0.1.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"planId": {
"$id": "/properties/planId",
"type": "integer",
"default": -1,
"minimum": 0
}
},
"required": [
"planId"
]
}

Binary file added documents/interface/atmosphere_tma-e_schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 25 additions & 0 deletions documents/interface/atmosphere_tma-p_schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"title": "tma-ac_schema_0_1",

"$id": "http://atmosphere-eubrazil.eu/tma-_aschema_v0.1.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"metricId": {
"$id": "/properties/metricId",
"type": "integer",
"default": -1,
"minimum": 0
},
"timestamp": {
"$id": "/properties/timestamp",
"type": "integer",
"default": -1,
"minimum": 0
}
},
"required": [
"metricId",
"timestamp"
]
}
Binary file added documents/interface/atmosphere_tma-p_schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion tma-framework-a
Submodule tma-framework-a updated 42 files
+4 −0 .gitignore
+2 −0 analyzer/dashboard/.dockerignore
+2 −0 analyzer/dashboard/Dockerfile
+197 −0 analyzer/dashboard/chart.js
+462 −0 analyzer/dashboard/configurations.htm
+12 −0 analyzer/dashboard/index.htm
+46 −0 analyzer/dashboard/properties-chart.htm
+38 −0 analyzer/dashboard/trustworthiness-chart.htm
+44 −0 analyzer/src/main/java/eubrazil/atmosphere/controller/DashboardController.java
+87 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Action.java
+83 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Actuator.java
+69 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Configuration.java
+84 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Data.java
+84 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/DataPK.java
+99 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Description.java
+122 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Probe.java
+53 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Properties.java
+122 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/Resource.java
+25 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/dashboard/Periodicity.java
+25 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/dashboard/PropRelevance.java
+48 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/dashboard/Properties.java
+35 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/dashboard/PropertiesJsonMap.java
+25 −0 analyzer/src/main/java/eubrazil/atmosphere/entity/dashboard/Thresholds.java
+31 −0 analyzer/src/main/java/eubrazil/atmosphere/exceptions/UndefinedMetricException.java
+4 −1 analyzer/src/main/java/eubrazil/atmosphere/qualitymodel/Attribute.java
+7 −7 analyzer/src/main/java/eubrazil/atmosphere/qualitymodel/CompositeAttribute.java
+26 −11 analyzer/src/main/java/eubrazil/atmosphere/qualitymodel/ConfigurationProfile.java
+147 −65 analyzer/src/main/java/eubrazil/atmosphere/qualitymodel/LeafAttribute.java
+69 −24 analyzer/src/main/java/eubrazil/atmosphere/qualitymodel/Metric.java
+208 −0 analyzer/src/main/java/eubrazil/atmosphere/qualitymodel/MetricNormalizationKind.java
+11 −0 analyzer/src/main/java/eubrazil/atmosphere/repository/DashboardRepository.java
+31 −0 analyzer/src/main/java/eubrazil/atmosphere/repository/KnowledgeRepository.java
+19 −0 analyzer/src/main/java/eubrazil/atmosphere/service/DashboardService.java
+3 −1 analyzer/src/main/resources/application.properties
+120 −0 analyzer/src/main/resources/db/migration/V1__bd_init.sql
+ architecture/diagrams/TMA-A/DependabilityQualityModel.png
+ architecture/diagrams/TMA-A/PrivacyQualityModel.png
+ architecture/diagrams/TMA-A/SecurityQualityModel.png
+217 −76 architecture/metamodels/qualitymodel.aird
+45 −19 architecture/metamodels/qualitymodel.ecore
+39 −17 architecture/metamodels/qualitymodel.genmodel
+ architecture/metamodels/qualitymodel.png
2 changes: 1 addition & 1 deletion tma-framework-e
Submodule tma-framework-e updated 42 files
+25 −0 development/actuators/.gitignore
+ development/actuators/.mvn/wrapper/maven-wrapper.jar
+1 −0 development/actuators/.mvn/wrapper/maven-wrapper.properties
+21 −0 development/actuators/README.md
+27 −0 development/actuators/demo-actuator-java/.gitignore
+ development/actuators/demo-actuator-java/.mvn/wrapper/maven-wrapper.jar
+1 −0 development/actuators/demo-actuator-java/.mvn/wrapper/maven-wrapper.properties
+20 −0 development/actuators/demo-actuator-java/README.md
+61 −0 development/actuators/demo-actuator-java/pom.xml
+31 −0 development/actuators/demo-actuator-java/src/main/java/eubr/atmosphere/tma/actuator/ActuatorAPI.java
+29 −0 development/actuators/demo-actuator-java/src/main/java/eubr/atmosphere/tma/actuator/ConcreteActuator.java
+12 −0 ...opment/actuators/demo-actuator-java/src/main/java/eubr/atmosphere/tma/actuator/DemoActuatorApplication.java
+11 −0 development/actuators/demo-actuator-java/src/main/resources/actuator-payload.json
+3 −0 development/actuators/demo-actuator-java/src/main/resources/application.properties
+ development/actuators/demo-actuator-java/src/main/resources/keys/priv-key-actuator
+ development/actuators/demo-actuator-java/src/main/resources/keys/pub-key-actuator
+ development/actuators/demo-actuator-java/src/main/resources/keys/pub-key-executor
+27 −0 development/actuators/kubernetes-actuator/.gitignore
+ development/actuators/kubernetes-actuator/.mvn/wrapper/maven-wrapper.jar
+1 −0 development/actuators/kubernetes-actuator/.mvn/wrapper/maven-wrapper.properties
+62 −0 development/actuators/kubernetes-actuator/pom.xml
+49 −0 development/actuators/kubernetes-actuator/src/main/java/eubr/atmosphere/tma/actuator/KubernetesActuator.java
+12 −0 ...actuators/kubernetes-actuator/src/main/java/eubr/atmosphere/tma/actuator/KubernetesActuatorApplication.java
+92 −0 ...lopment/actuators/kubernetes-actuator/src/main/java/eubr/atmosphere/tma/actuator/services/RestServices.java
+11 −0 development/actuators/kubernetes-actuator/src/main/resources/actuator-payload.json
+3 −0 development/actuators/kubernetes-actuator/src/main/resources/application.properties
+ development/actuators/kubernetes-actuator/src/main/resources/keys/priv-key-actuator
+ development/actuators/kubernetes-actuator/src/main/resources/keys/pub-key-actuator
+ development/actuators/kubernetes-actuator/src/main/resources/keys/pub-key-executor
+4 −0 development/libraries/java-actuator-base/.gitignore
+43 −0 development/libraries/java-actuator-base/README.md
+41 −0 development/libraries/java-actuator-base/pom.xml
+6 −0 development/libraries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/Actuator.java
+57 −0 development/libraries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/ActuatorPayload.java
+35 −0 development/libraries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/Configuration.java
+141 −0 development/libraries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/crypto/KeyManager.java
+105 −0 development/libraries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/filters/DecryptFilter.java
+32 −0 ...opment/libraries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/utils/PropertiesManager.java
+33 −0 ...ibraries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/wrappers/ByteArrayServletStream.java
+102 −0 ...va-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/wrappers/HttpServletRequestWritableWrapper.java
+76 −0 ...aries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/wrappers/HttpServletResponseCopier.java
+42 −0 ...aries/java-actuator-base/src/main/java/eubr/atmosphere/tma/actuator/wrappers/ServletOutputStreamCopier.java
2 changes: 1 addition & 1 deletion tma-framework-k
Submodule tma-framework-k updated 35 files
+6 −0 .gitignore
+2 −0 README.md
+6 −6 database/TMA-K_create_database.sql
+19 −18 development/README.md
+0 −14 development/TMA_Admin_Console/.gitignore
+0 −21 development/TMA_Admin_Console/pom.xml
+0 −185 development/TMA_Admin_Console/src/main/java/pt/uc/dei/eubr/atmosphere/tma/admin/AdminConsole.java
+0 −62 development/TMA_Admin_Console/src/main/java/pt/uc/dei/eubr/atmosphere/tma/admin/GenSig.java
+0 −24 development/TMA_Admin_Console/src/main/java/pt/uc/dei/eubr/atmosphere/tma/admin/database/ActuatorManager.java
+0 −94 development/TMA_Admin_Console/src/main/java/pt/uc/dei/eubr/atmosphere/tma/admin/database/DatabaseManager.java
+0 −30 development/TMA_Admin_Console/src/main/java/pt/uc/dei/eubr/atmosphere/tma/admin/database/ResourceManager.java
+5 −6 development/ceph/README.md
+10 −0 development/mysql/Dockerfile
+18 −10 development/mysql/README.md
+0 −0 development/mysql/TMA-K_insert_example_data.sql
+8 −0 development/mysql/build.sh
+4 −2 development/mysql/mysql-deployment.yaml
+22 −0 development/mysql/setup_database.sh
+1 −0 development/tma-admin-console/.gitignore
+20 −0 development/tma-admin-console/Dockerfile
+45 −0 development/tma-admin-console/README.md
+1 −0 development/tma-admin-console/build.sh
+153 −0 development/tma-admin-console/pom.xml
+1 −0 development/tma-admin-console/run.sh
+207 −0 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/AdminConsole.java
+54 −33 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/KeyManager.java
+1 −1 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/database/Action.java
+15 −10 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/database/ActionManager.java
+52 −0 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/database/ActuatorManager.java
+1 −1 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/database/Configuration.java
+103 −0 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/database/DatabaseManager.java
+37 −0 development/tma-admin-console/src/main/java/eu/atmosphere/tmaf/admin/database/ResourceManager.java
+46 −0 development/tma-admin-console/src/main/resources/log4j2.xml
+28 −0 development/tma-admin-console/tma-admin-console.yaml
+0 −0 interface/atmosphere_tma-k_actions-example.json
2 changes: 1 addition & 1 deletion tma-framework-m
Submodule tma-framework-m updated 47 files
+33 −34 development/dependency/docker-client-base/cert.pem
+33 −34 development/dependency/python-probe-base/cert.pem
+33 −34 development/libraries/cert.pem
+1 −1 development/libraries/monitor-client/src/main/java/eu/atmosphere/tmaf/monitor/client/MonitorClient.java
+1 −1 development/probes/README.md
+1 −1 development/probes/probe-docker-metrics/Dockerfile
+0 −1 development/probes/probe-docker-metrics/README
+12 −3 development/probes/probe-docker-metrics/README.md
+1 −1 development/probes/probe-docker-metrics/build.sh
+1 −1 development/probes/probe-docker-metrics/run.sh
+0 −2 development/probes/probe-k8-metrics/probe-k8-containers-metrics/build.sh
+0 −36 development/probes/probe-k8-metrics/probe-k8-containers-metrics/cert.pem
+1 −1 development/probes/probe-k8s-docker/Dockerfile
+16 −5 development/probes/probe-k8s-docker/README.md
+2 −0 development/probes/probe-k8s-docker/build.sh
+35 −0 development/probes/probe-k8s-docker/cert.pem
+0 −0 development/probes/probe-k8s-docker/data.py
+0 −0 development/probes/probe-k8s-docker/dockerAPI.py
+0 −0 development/probes/probe-k8s-docker/message.py
+0 −0 development/probes/probe-k8s-docker/observation.py
+5 −5 development/probes/probe-k8s-docker/probe-k8s-docker.yaml
+16 −0 development/probes/probe-k8s-metrics-server/.gitignore
+17 −0 development/probes/probe-k8s-metrics-server/Dockerfile
+46 −0 development/probes/probe-k8s-metrics-server/README.md
+1 −0 development/probes/probe-k8s-metrics-server/build.sh
+121 −0 development/probes/probe-k8s-metrics-server/pom.xml
+26 −0 development/probes/probe-k8s-metrics-server/probe-k8s-metrics-server.yaml
+237 −0 development/probes/probe-k8s-metrics-server/src/main/java/eubr/atmosphere/tma/probes/ProbeKubernetes.java
+32 −0 ...pment/probes/probe-k8s-metrics-server/src/main/java/eubr/atmosphere/tma/probes/utils/PropertiesManager.java
+35 −0 development/probes/probe-k8s-metrics-server/src/main/resources/data-nodes.json
+256 −0 development/probes/probe-k8s-metrics-server/src/main/resources/data-pods.json
+11 −0 development/probes/probe-k8s-metrics-server/src/main/resources/data-probe.sql
+2 −0 development/probes/probe-k8s-metrics-server/src/main/resources/environment.properties
+31 −0 development/probes/probe-k8s-metrics-server/src/main/resources/log4j2.xml
+7 −0 development/probes/probe-k8s-network/README.md
+1 −1 development/probes/probe-python-demo/Dockerfile
+17 −3 development/probes/probe-python-demo/README.md
+1 −1 development/probes/probe-python-demo/build.sh
+1 −1 development/probes/probe-python-demo/probe-python-demo.yaml
+1 −1 development/server/README.md
+1 −1 development/server/kafka/README.md
+1 −1 development/server/monitor-server-python/README.md
+2 −0 development/server/monitor-server-python/monitor-api-python.yaml
+33 −34 development/server/monitor-server-python/monitor-api-python/cert.pem
+50 −50 development/server/monitor-server-python/monitor-api-python/key.pem
+33 −34 development/test/testing-json-format/cert.pem
+1 −1 development/test/testing-json-format/testing-json-format.sh

0 comments on commit 925fd92

Please sign in to comment.