The integrity of your deployments is our paramount commitment. We are devoted to strengthening our Kubernetes platform to comply with the most stringent security standards. Trust is the bedrock of our relationships, and we manifest this commitment by undergoing rigorous third-party audits to ensure compliance. We pledge unwavering support as you manage and deploy solutions within your environment, emphasizing security and reliability. Examine our compliance with various frameworks, laws, and regulations to understand our dedication to upholding robust security standards for the solutions you manage and deploy.
\ No newline at end of file
+ Compliance - EPAM Delivery Platform
The integrity of your deployments is our paramount commitment. We are devoted to strengthening our Kubernetes platform to comply with the most stringent security standards. Trust is the bedrock of our relationships, and we manifest this commitment by undergoing rigorous third-party audits to ensure compliance. We pledge unwavering support as you manage and deploy solutions within your environment, emphasizing security and reliability. Examine our compliance with various frameworks, laws, and regulations to understand our dedication to upholding robust security standards for the solutions you manage and deploy.
\ No newline at end of file
diff --git a/developer-guide/annotations-and-labels/index.html b/developer-guide/annotations-and-labels/index.html
index 007403fc8..0ebf8e975 100644
--- a/developer-guide/annotations-and-labels/index.html
+++ b/developer-guide/annotations-and-labels/index.html
@@ -1,4 +1,4 @@
- Annotations and Labels - EPAM Delivery Platform
EPAM Delivery Platform uses labels to interact with various resources in a Kubernetes cluster. This guide details the resources, annotations, and labels used by the platform to streamline operations, enhance monitoring, and enforce governance.
EPAM Delivery Platform uses labels to interact with various resources in a Kubernetes cluster. This guide details the resources, annotations, and labels used by the platform to streamline operations, enhance monitoring, and enforce governance.
In EPAM Delivery Platform, we guarantee the quality of the product not only by using the most advanced tools and best practices but also by covering the whole product functionality with our dedicated automated tests.
Autotests are significant part of our verification flow. Continuous improvement of the verification mechanisms quality is performed to provide users with the most stable version of the platform.
The autotest coverage status is presented on the scheme below:
In our testing flow, each release is verified by the following tests:
Test Group
Description
What's Covered
API Tests
Tekton Gerrit, GitHub, and GitLab API long regression
Codebase provisioning, reviewing and building pipelines, adding new branches, deploying applications (in a custom namespace), Jira integration, and rechecking for review pipeline.
UI Tests
Tekton Gerrit, GitHub, and GitLab UI long regression
Codebase provisioning, reviewing and building pipelines, adding new branches, deploying applications (in a custom namespace), Jira integration, and rechecking for review pipeline.
Short Tests
Tekton Gerrit , GitHub, and GitLab API short regression
Codebase provisioning, reviewing and building pipelines, deploying applications (in a custom namespace), rechecking for review pipeline
Smoke
Tekton Gerrit Smoke
Codebase provisioning, reviewing and building pipelines, deploying applications.
In EPAM Delivery Platform, we guarantee the quality of the product not only by using the most advanced tools and best practices but also by covering the whole product functionality with our dedicated automated tests.
Autotests are significant part of our verification flow. Continuous improvement of the verification mechanisms quality is performed to provide users with the most stable version of the platform.
The autotest coverage status is presented on the scheme below:
In our testing flow, each release is verified by the following tests:
Test Group
Description
What's Covered
API Tests
Tekton Gerrit, GitHub, and GitLab API long regression
Codebase provisioning, reviewing and building pipelines, adding new branches, deploying applications (in a custom namespace), Jira integration, and rechecking for review pipeline.
UI Tests
Tekton Gerrit, GitHub, and GitLab UI long regression
Codebase provisioning, reviewing and building pipelines, adding new branches, deploying applications (in a custom namespace), Jira integration, and rechecking for review pipeline.
Short Tests
Tekton Gerrit , GitHub, and GitLab API short regression
Codebase provisioning, reviewing and building pipelines, deploying applications (in a custom namespace), rechecking for review pipeline
Smoke
Tekton Gerrit Smoke
Codebase provisioning, reviewing and building pipelines, deploying applications.
This document describes the EPAM Delivery Platform (EDP) deployment architecture on AWS. It utilizes various AWS services such as Amazon Elastic Kubernetes Service (EKS), Amazon EC2, Amazon Route 53, and others to build and deploy software in a repeatable, automated way.
The EDP deployment architecture consists of two AWS accounts: Shared and Explorer. The Shared account hosts shared services, while the Explorer account runs the development team workload and EDP services. Both accounts have an AWS EKS cluster deployed in multiple Availability Zones (AZs). The EKS cluster runs the EDP Services, development team workload, and shared services in the case of the Shared account.
AWS Elastic Kubernetes Service (EKS): A managed Kubernetes service used to run the EDP Services, development team workload, and shared services. EKS provides easy deployment and management of Kubernetes clusters.
Amazon EC2: Instances running within private subnets that serve as nodes for the EKS cluster. Autoscaling Groups are used to deploy these instances, allowing for scalability based on demand.
Amazon Route 53: A DNS web service manages external and internal DNS records for the EDP deployment. It enables easy access to resources using user-friendly domain names.
AWS Application Load Balancer (ALB): Used for managing ingress traffic into the EDP deployment. Depending on requirements, ALBs can be configured as internal or external load balancers.
AWS WAF: Web Application Firewall service used to protect external ALBs from common web exploits by filtering malicious requests.
AWS Certificate Manager (ACM): A service that provisions manages, and deploys SSL/TLS certificates for use with AWS services. ACM is used to manage SSL certificates for secure communication within the EDP deployment.
AWS Elastic Container Registry (ECR): A fully-managed Docker container registry that stores and manages Docker images. ECR provides a secure and scalable solution for storing container images used in the EDP deployment.
AWS Systems Manager Parameter Store: Used to securely store and manage secrets required by various components of the EDP deployment. Parameter Store protects sensitive information such as API keys, database credentials, and other secrets.
The EKS cluster is deployed across multiple AZs to ensure high availability and fault tolerance. This allows for automatic failover in case of an AZ outage or instance failure. Autoscaling Groups automatically adjust the number of EC2 instances based on demand, ensuring scalability while maintaining availability.
The EPAM Delivery Platform (EDP) deployment architecture on AWS follows best practices and patterns from the Well-Architected Framework. By leveraging AWS services such as EKS, EC2, Route 53, ALB, WAF, ACM, and Parameter Store, the EDP provides a robust and scalable CI/CD system that enables developers to deploy and manage infrastructure and applications quickly. The architecture ensures high availability, fault tolerance, reliability, performance efficiency, security, and cost optimization for the EDP deployment.
\ No newline at end of file
+ EDP Deployment on AWS - EPAM Delivery Platform
This document describes the EPAM Delivery Platform (EDP) deployment architecture on AWS. It utilizes various AWS services such as Amazon Elastic Kubernetes Service (EKS), Amazon EC2, Amazon Route 53, and others to build and deploy software in a repeatable, automated way.
The EDP deployment architecture consists of two AWS accounts: Shared and Explorer. The Shared account hosts shared services, while the Explorer account runs the development team workload and EDP services. Both accounts have an AWS EKS cluster deployed in multiple Availability Zones (AZs). The EKS cluster runs the EDP Services, development team workload, and shared services in the case of the Shared account.
AWS Elastic Kubernetes Service (EKS): A managed Kubernetes service used to run the EDP Services, development team workload, and shared services. EKS provides easy deployment and management of Kubernetes clusters.
Amazon EC2: Instances running within private subnets that serve as nodes for the EKS cluster. Autoscaling Groups are used to deploy these instances, allowing for scalability based on demand.
Amazon Route 53: A DNS web service manages external and internal DNS records for the EDP deployment. It enables easy access to resources using user-friendly domain names.
AWS Application Load Balancer (ALB): Used for managing ingress traffic into the EDP deployment. Depending on requirements, ALBs can be configured as internal or external load balancers.
AWS WAF: Web Application Firewall service used to protect external ALBs from common web exploits by filtering malicious requests.
AWS Certificate Manager (ACM): A service that provisions manages, and deploys SSL/TLS certificates for use with AWS services. ACM is used to manage SSL certificates for secure communication within the EDP deployment.
AWS Elastic Container Registry (ECR): A fully-managed Docker container registry that stores and manages Docker images. ECR provides a secure and scalable solution for storing container images used in the EDP deployment.
AWS Systems Manager Parameter Store: Used to securely store and manage secrets required by various components of the EDP deployment. Parameter Store protects sensitive information such as API keys, database credentials, and other secrets.
The EKS cluster is deployed across multiple AZs to ensure high availability and fault tolerance. This allows for automatic failover in case of an AZ outage or instance failure. Autoscaling Groups automatically adjust the number of EC2 instances based on demand, ensuring scalability while maintaining availability.
The EPAM Delivery Platform (EDP) deployment architecture on AWS follows best practices and patterns from the Well-Architected Framework. By leveraging AWS services such as EKS, EC2, Route 53, ALB, WAF, ACM, and Parameter Store, the EDP provides a robust and scalable CI/CD system that enables developers to deploy and manage infrastructure and applications quickly. The architecture ensures high availability, fault tolerance, reliability, performance efficiency, security, and cost optimization for the EDP deployment.
\ No newline at end of file
diff --git a/developer-guide/aws-infrastructure-cost-estimation/index.html b/developer-guide/aws-infrastructure-cost-estimation/index.html
index cbb30d24d..58ac095dd 100644
--- a/developer-guide/aws-infrastructure-cost-estimation/index.html
+++ b/developer-guide/aws-infrastructure-cost-estimation/index.html
@@ -1 +1 @@
- AWS Infrastructure Cost Estimation - EPAM Delivery Platform
Effective planning and budgeting are essential for developing applications in cloud computing, with a key part being accurate infrastructure cost estimation. This not only helps in keeping within budget but also enables informed decision-making and resource optimization for project viability.
This guide aims to offer an in-depth look at the factors affecting AWS infrastructure costs for KubeRocketCI and includes analytics and tools for cost estimation.
This section contains tables outlining the key components of our AWS infrastructure, including a brief description of each component's role, its purpose within our infrastructure, and an estimate of its monthly cost.
Note
The costs mentioned below are estimates. For the most accurate and up-to-date pricing, please refer to the AWS official documentation.
The table below outlines key AWS infrastructure components for KubeRocketCI, detailing each component's role, purpose, and estimated monthly cost:
Component
Description
Purpose Within Infrastructure
Application Load Balancer (ALB)
Distributes incoming application traffic across multiple targets.
Ensures high availability and fault tolerance for our applications.
Virtual Private Cloud (VPC)
Provides an isolated section of the AWS cloud where resources can be launched.
Segregates our infrastructure for enhanced security and management.
3x Network Address Translation (NAT) Gateways
Enables instances in a private subnet to connect to the internet or other AWS services.
Provides internet access to EC2 instances without exposing them to the public internet.
Elastic Container Registry (ECR)
A fully managed container registry.
Stores, manages, and deploys container images.
Elastic Kubernetes Service (EKS)
A managed Kubernetes service.
Simplifies running Kubernetes applications on AWS.
Elastic Block Store (EBS)
Provides persistent block storage volumes for use with EC2 instances.
Offers highly available and durable storage for our applications.
To further assist in your planning and budgeting efforts, we have pre-configured the AWS Pricing Calculator with inputs matching our infrastructure setup. This tool allows you to explore and adjust the cost estimation based on your specific needs, giving you a personalized overview of potential expenses.
Access the AWS Pricing Calculator with our pre-configured setup here: AWS Pricing Calculator
Effective planning and budgeting are essential for developing applications in cloud computing, with a key part being accurate infrastructure cost estimation. This not only helps in keeping within budget but also enables informed decision-making and resource optimization for project viability.
This guide aims to offer an in-depth look at the factors affecting AWS infrastructure costs for KubeRocketCI and includes analytics and tools for cost estimation.
This section contains tables outlining the key components of our AWS infrastructure, including a brief description of each component's role, its purpose within our infrastructure, and an estimate of its monthly cost.
Note
The costs mentioned below are estimates. For the most accurate and up-to-date pricing, please refer to the AWS official documentation.
The table below outlines key AWS infrastructure components for KubeRocketCI, detailing each component's role, purpose, and estimated monthly cost:
Component
Description
Purpose Within Infrastructure
Application Load Balancer (ALB)
Distributes incoming application traffic across multiple targets.
Ensures high availability and fault tolerance for our applications.
Virtual Private Cloud (VPC)
Provides an isolated section of the AWS cloud where resources can be launched.
Segregates our infrastructure for enhanced security and management.
3x Network Address Translation (NAT) Gateways
Enables instances in a private subnet to connect to the internet or other AWS services.
Provides internet access to EC2 instances without exposing them to the public internet.
Elastic Container Registry (ECR)
A fully managed container registry.
Stores, manages, and deploys container images.
Elastic Kubernetes Service (EKS)
A managed Kubernetes service.
Simplifies running Kubernetes applications on AWS.
Elastic Block Store (EBS)
Provides persistent block storage volumes for use with EC2 instances.
Offers highly available and durable storage for our applications.
To further assist in your planning and budgeting efforts, we have pre-configured the AWS Pricing Calculator with inputs matching our infrastructure setup. This tool allows you to explore and adjust the cost estimation based on your specific needs, giving you a personalized overview of potential expenses.
Access the AWS Pricing Calculator with our pre-configured setup here: AWS Pricing Calculator
The reference architecture of the EPAM Delivery Platform (EDP) on AWS is designed to provide a robust and scalable CI/CD system for developing and deploying software in a repeatable and automated manner. The architecture leverages AWS Managed Services to enable developers to quickly deploy and manage infrastructure and applications. EDP recommends to follow the best practices and patterns from the Well-Architected Framework, the AWS Architecture Center, and EKS Best Practices Guide.
The AWS Cloud comprises three accounts: Production, Shared, and Development.
Note
AWS Account management is out of scope for this document.
Each account serves specific purposes:
The Production account is used to host production workloads. The Production account serves as the final destination for deploying business applications. It maintains a separate ECR registry to store Docker images for production-level applications. The environment is designed to be highly resilient and scalable, leveraging the EPAM Delivery Platform's CI/CD pipeline to ensure consistent and automated deployments. With proper access control and separation from development environments, the Production account provides a stable and secure environment for running mission-critical applications.
The Development account is dedicated to development workload and lower environments. This account hosts the EDP itself, running on AWS EKS. It provides developers an isolated environment to build, test, and deploy their applications in lower environments, ensuring separation from production workloads. Developers can connect to the AWS Cloud using a VPN, enforcing secure access.
The Shared holds shared services that are accessible to all accounts within the organization. These services include SonarQube, Nexus, and Keycloak, which are deployed in Kubernetes Clusters managed by AWS Elastic Kubernetes Service (EKS). The shared services leverage AWS RDS, AWS EFS, and AWS ALB/NLB. The deployment of the shared services is automated using Kubernetes cluster-addons approach with GitOps and Argo CD.
EPAM Delivery Platform Reference Architecture on AWS
Infrastructure as Code (IaC) is a key principle in the EPAM Delivery Platform architecture. Terraform is the IaC tool to provision and manage all services in each account. AWS S3 and AWS DynamoDB serve as the backend for Terraform state, ensuring consistency and reliability in the deployment process. This approach enables the architecture to be version-controlled and allows for easy replication and reproducibility of environments.
The architecture utilizes AWS Elastic Container Registry (ECR) as a Docker Registry for container image management. ECR offers a secure, scalable, and reliable solution for storing and managing container images. It integrates seamlessly with other AWS services and provides a highly available and durable storage solution for containers in the CI/CD pipeline.
The EPAM Delivery Platform implements IAM Roles for Service Accounts (IRSA) to provide secure access to AWS services from Kubernetes Clusters. This feature enables fine-grained access control with individual Kubernetes pods assuming specific IAM roles for authenticated access to AWS resources. IRSA eliminates the need for managing and distributing access keys within the cluster, significantly enhancing security and reducing operational complexity.
The architecture uses the AWS Certificate Manager (ACM) to secure communication between services to provide SSL certificates. ACM eliminates the need to manually manage SSL/TLS certificates, automating the renewal and deployment process. The EDP ensures secure and encrypted traffic within its environment by leveraging ACM.
The architecture's external Application Load Balancer (ALB) endpoint is protected by the AWS Web Application Firewall (WAF). WAF protects against common web exploits and ensures the security and availability of the applications hosted within the EDP. It offers regular rule updates and easy integration with other AWS services.
The architecture leverages the AWS Systems Manager Parameter Store and Secrets Manager to securely store and manage all secrets and parameters utilized within the EKS clusters—parameter Store stores general configuration information, such as database connection strings and API keys. In contrast, Secrets Manager securely stores sensitive information, such as passwords and access tokens. By centralizing secrets management, the architecture ensures proper access control and reduces the risk of unauthorized access.
For observability and monitoring, the EDP leverages a suite of AWS Managed Services designed to provide comprehensive insights into the performance and health of applications and infrastructure:
AWS CloudWatch is utilized for monitoring and observability, offering detailed insights into application and infrastructure performance. It enables real-time monitoring of logs, metrics, and events, facilitating proactive issue resolution and performance optimization.
AWS OpenSearch Service (successor to Amazon Elasticsearch Service) provides powerful search and analytics capabilities. It allows for the analysis of log data and metrics, supporting enhanced application monitoring and user experience optimization.
AWS Managed Grafana offers a scalable, secure, and fully managed Grafana service, enabling developers to create and share dashboards for visualizing real-time data.
AWS Prometheus Service, a managed Prometheus-compatible monitoring service, is used for monitoring Kubernetes and container environments. It supports powerful queries and provides detailed insights into container and microservices architectures.
The reference architecture of the EPAM Delivery Platform on AWS provides a comprehensive and scalable environment for building and deploying software applications. With a strong focus on automation, security, and best practices, this architecture enables developers to leverage the full potential of AWS services while following industry-standard DevOps practices.
\ No newline at end of file
+ EDP Reference Architecture on AWS - EPAM Delivery Platform
The reference architecture of the EPAM Delivery Platform (EDP) on AWS is designed to provide a robust and scalable CI/CD system for developing and deploying software in a repeatable and automated manner. The architecture leverages AWS Managed Services to enable developers to quickly deploy and manage infrastructure and applications. EDP recommends to follow the best practices and patterns from the Well-Architected Framework, the AWS Architecture Center, and EKS Best Practices Guide.
The AWS Cloud comprises three accounts: Production, Shared, and Development.
Note
AWS Account management is out of scope for this document.
Each account serves specific purposes:
The Production account is used to host production workloads. The Production account serves as the final destination for deploying business applications. It maintains a separate ECR registry to store Docker images for production-level applications. The environment is designed to be highly resilient and scalable, leveraging the EPAM Delivery Platform's CI/CD pipeline to ensure consistent and automated deployments. With proper access control and separation from development environments, the Production account provides a stable and secure environment for running mission-critical applications.
The Development account is dedicated to development workload and lower environments. This account hosts the EDP itself, running on AWS EKS. It provides developers an isolated environment to build, test, and deploy their applications in lower environments, ensuring separation from production workloads. Developers can connect to the AWS Cloud using a VPN, enforcing secure access.
The Shared holds shared services that are accessible to all accounts within the organization. These services include SonarQube, Nexus, and Keycloak, which are deployed in Kubernetes Clusters managed by AWS Elastic Kubernetes Service (EKS). The shared services leverage AWS RDS, AWS EFS, and AWS ALB/NLB. The deployment of the shared services is automated using Kubernetes cluster-addons approach with GitOps and Argo CD.
EPAM Delivery Platform Reference Architecture on AWS
Infrastructure as Code (IaC) is a key principle in the EPAM Delivery Platform architecture. Terraform is the IaC tool to provision and manage all services in each account. AWS S3 and AWS DynamoDB serve as the backend for Terraform state, ensuring consistency and reliability in the deployment process. This approach enables the architecture to be version-controlled and allows for easy replication and reproducibility of environments.
The architecture utilizes AWS Elastic Container Registry (ECR) as a Docker Registry for container image management. ECR offers a secure, scalable, and reliable solution for storing and managing container images. It integrates seamlessly with other AWS services and provides a highly available and durable storage solution for containers in the CI/CD pipeline.
The EPAM Delivery Platform implements IAM Roles for Service Accounts (IRSA) to provide secure access to AWS services from Kubernetes Clusters. This feature enables fine-grained access control with individual Kubernetes pods assuming specific IAM roles for authenticated access to AWS resources. IRSA eliminates the need for managing and distributing access keys within the cluster, significantly enhancing security and reducing operational complexity.
The architecture uses the AWS Certificate Manager (ACM) to secure communication between services to provide SSL certificates. ACM eliminates the need to manually manage SSL/TLS certificates, automating the renewal and deployment process. The EDP ensures secure and encrypted traffic within its environment by leveraging ACM.
The architecture's external Application Load Balancer (ALB) endpoint is protected by the AWS Web Application Firewall (WAF). WAF protects against common web exploits and ensures the security and availability of the applications hosted within the EDP. It offers regular rule updates and easy integration with other AWS services.
The architecture leverages the AWS Systems Manager Parameter Store and Secrets Manager to securely store and manage all secrets and parameters utilized within the EKS clusters—parameter Store stores general configuration information, such as database connection strings and API keys. In contrast, Secrets Manager securely stores sensitive information, such as passwords and access tokens. By centralizing secrets management, the architecture ensures proper access control and reduces the risk of unauthorized access.
For observability and monitoring, the EDP leverages a suite of AWS Managed Services designed to provide comprehensive insights into the performance and health of applications and infrastructure:
AWS CloudWatch is utilized for monitoring and observability, offering detailed insights into application and infrastructure performance. It enables real-time monitoring of logs, metrics, and events, facilitating proactive issue resolution and performance optimization.
AWS OpenSearch Service (successor to Amazon Elasticsearch Service) provides powerful search and analytics capabilities. It allows for the analysis of log data and metrics, supporting enhanced application monitoring and user experience optimization.
AWS Managed Grafana offers a scalable, secure, and fully managed Grafana service, enabling developers to create and share dashboards for visualizing real-time data.
AWS Prometheus Service, a managed Prometheus-compatible monitoring service, is used for monitoring Kubernetes and container environments. It supports powerful queries and provides detailed insights into container and microservices architectures.
The reference architecture of the EPAM Delivery Platform on AWS provides a comprehensive and scalable environment for building and deploying software applications. With a strong focus on automation, security, and best practices, this architecture enables developers to leverage the full potential of AWS services while following industry-standard DevOps practices.
\ No newline at end of file
diff --git a/developer-guide/edp-workflow/index.html b/developer-guide/edp-workflow/index.html
index b7414d18b..c51db31cf 100644
--- a/developer-guide/edp-workflow/index.html
+++ b/developer-guide/edp-workflow/index.html
@@ -1,4 +1,4 @@
- KubeRocketCI Project Rules. Working Process - EPAM Delivery Platform
This page contains the details on the project rules and working process for KubeRocketCI team and contributors. Explore the main points about working with GitHub, following the main commit flow, as well as the details about commit types and message below.
Before starting the development, please check the project rules:
It is highly recommended to become familiar with the GitHub flow. For details, please refer to the GitHub official documentation and pay attention to the main points:
a. Creating pull requests in GitHub.
b. Resolution of Merge Conflict.
c. Comments resolution.
d. One GitHub task should have one Pull Request (PR) if it doesn't change multiple operators. If there are many changes within one PR, amend the commit.
Only the Assignee is responsible for the PR merger and Jira task status.
With KubeRocketCI, the main workflow is based on the getting a Jira task and creating a Pull Request according to the rules described below.
Workflow
Get Jira task → implement, verify by yourself the results → create Pull Request (PR) → send for review → resolve comments/add changes, ask colleagues for the final review → track the PR merge → verify by yourself the results → change the status in the Jira ticket to CODE COMPLETE or RESOLVED → share necessary links with a QA specialist in the QA Verification channel → QA specialist closes the Jira task after his verification → Jira task should be CLOSED.
Commit Flow
Get a task in the Jira/GitHub dashboard. Please be aware of the following points:
a. Every task has a reporter who can provide more details in case something is not clear.
b. The responsible person for the task and code implementation is the assignee who tracks the following:
Actual Jira task status.
Time logging.
Add comments, attach necessary files.
In comments, add link that refers to the merged PR (optional, if not related to many repositories).
Code review and the final merge.
MS Teams chats - ping other colleagues, answer questions, etc.
Verification by a QA specialist.
Bug fixing.
c. Pay attention to the task Status that differs in different entities, the workflow will help to see the whole task processing:
View Jira workflow
d. There are several entities that are used on the KubeRocketCI project: Story, Improvement, Task, Bug.
a. Every task has a reporter who can provide more details in case something is not clear.
b. The responsible person for the task and code implementation is the assignee who tracks the following:
Actual GitHub task status.
Add comments, attach necessary files.
In comments, add link that refers to the merged PR (optional, if not related to many repositories).
Code review and the final merge.
MS Teams chats - ping other colleagues, answer questions, etc.
Verification by a QA specialist.
Bug fixing.
c. If the task is created on your own, make sure it is populated completely. See an example below:
GitHub issue
Implement feature, improvement, fix and check the results on your own. If it is impossible to check the results of your work before the merge, verify all later.
When committing, use the pattern: commit type: Commit message (#GitHub ticket number).
a. commit type:
feat: (new feature for the user, not a new feature for build script)
fix: (bug fix for the user, not a fix to a build script)
docs: (changes to the documentation)
style: (formatting, missing semicolons, etc; no production code change)
refactor: (refactoring production code, eg. renaming a variable)
test: (adding missing tests, refactoring tests; no production code change)
chore: (updating grunt tasks etc; no production code change)
!: (added to other commit types to mark breaking changes) For example:
feat!: Add ingress links column into Applications table on stage page (#77)
+ KubeRocketCI Project Rules. Working Process - EPAM Delivery Platform
This page contains the details on the project rules and working process for KubeRocketCI team and contributors. Explore the main points about working with GitHub, following the main commit flow, as well as the details about commit types and message below.
Before starting the development, please check the project rules:
It is highly recommended to become familiar with the GitHub flow. For details, please refer to the GitHub official documentation and pay attention to the main points:
a. Creating pull requests in GitHub.
b. Resolution of Merge Conflict.
c. Comments resolution.
d. One GitHub task should have one Pull Request (PR) if it doesn't change multiple operators. If there are many changes within one PR, amend the commit.
Only the Assignee is responsible for the PR merger and Jira task status.
With KubeRocketCI, the main workflow is based on the getting a Jira task and creating a Pull Request according to the rules described below.
Workflow
Get Jira task → implement, verify by yourself the results → create Pull Request (PR) → send for review → resolve comments/add changes, ask colleagues for the final review → track the PR merge → verify by yourself the results → change the status in the Jira ticket to CODE COMPLETE or RESOLVED → share necessary links with a QA specialist in the QA Verification channel → QA specialist closes the Jira task after his verification → Jira task should be CLOSED.
Commit Flow
Get a task in the Jira/GitHub dashboard. Please be aware of the following points:
a. Every task has a reporter who can provide more details in case something is not clear.
b. The responsible person for the task and code implementation is the assignee who tracks the following:
Actual Jira task status.
Time logging.
Add comments, attach necessary files.
In comments, add link that refers to the merged PR (optional, if not related to many repositories).
Code review and the final merge.
MS Teams chats - ping other colleagues, answer questions, etc.
Verification by a QA specialist.
Bug fixing.
c. Pay attention to the task Status that differs in different entities, the workflow will help to see the whole task processing:
View Jira workflow
d. There are several entities that are used on the KubeRocketCI project: Story, Improvement, Task, Bug.
a. Every task has a reporter who can provide more details in case something is not clear.
b. The responsible person for the task and code implementation is the assignee who tracks the following:
Actual GitHub task status.
Add comments, attach necessary files.
In comments, add link that refers to the merged PR (optional, if not related to many repositories).
Code review and the final merge.
MS Teams chats - ping other colleagues, answer questions, etc.
Verification by a QA specialist.
Bug fixing.
c. If the task is created on your own, make sure it is populated completely. See an example below:
GitHub issue
Implement feature, improvement, fix and check the results on your own. If it is impossible to check the results of your work before the merge, verify all later.
When committing, use the pattern: commit type: Commit message (#GitHub ticket number).
a. commit type:
feat: (new feature for the user, not a new feature for build script)
fix: (bug fix for the user, not a fix to a build script)
docs: (changes to the documentation)
style: (formatting, missing semicolons, etc; no production code change)
refactor: (refactoring production code, eg. renaming a variable)
test: (adding missing tests, refactoring tests; no production code change)
chore: (updating grunt tasks etc; no production code change)
!: (added to other commit types to mark breaking changes) For example:
feat!: Add ingress links column into Applications table on stage page (#77)
BREAKING CHANGE: Ingress links column has been added into the Applications table on the stage details page
b. Commit message:
brief, for example:
fix: Remove secretKey duplication from registry secrets (#63)
or
descriptive, for example:
feat: Provide the ability to configure hadolint check (#88)
* Add configuration files .hadolint.yaml and .hadolint.yml to stash
Note
It is mandatory to start a commit message from a capital letter.
c. GitHub tickets are typically identified using a number preceded by the # sign and enclosed in parentheses.
Note
Make sure there is a descriptive commit message for a breaking change Pull Request. For example:
feat!: Add ingress links column into Applications table on stage page (#77)
BREAKING CHANGE: Ingress links column has been added into the Applications table on the stage details page
Create a Pull Request, for details, please refer to the Code Review Process:
GitHub issue
Note
If a Pull Request contains both new functionality and breaking changes, make sure the functionality description is placed before the breaking changes. For example:
feat!: Update Gerrit to improve access
Implement Developers group creation process
Align group permissions
BREAKING CHANGES: Update Gerrit config according to groups
The EPAM Delivery Platform (EDP) Developer Guide serves as a comprehensive technical resource specifically designed for developers. It offers detailed insights into expanding the functionalities of EDP. This section focuses on explaining the development approach and fundamental architectural blueprints that form the basis of the platform's ecosystem.
Within these pages, you'll find architectural diagrams, component schemas, and deployment strategies essential for grasping the structural elements of EDP. These technical illustrations serve as references, providing a detailed understanding of component interactions and deployment methodologies. Understanding the architecture of EDP and integrating third-party solutions into its established framework enables the creation of efficient, scalable, and customizable solutions within the EPAM Delivery Platform.
The diagram below illustrates how GitHub repositories and Docker registries are interconnected within the EDP ecosystem.
As a publicly available product, the EPAM Delivery Platform relies on various channels to share information, gather feedback, and distribute new releases effectively. This section outlines the diverse channels through which users can engage with our platform and stay informed about the latest developments and enhancements.
Our product is presented on AWS and Civo marketplaces. It's essential to ensure that the product information on these platforms is up-to-date and accurately reflects the latest version of our software:
Our platform components, optional enhancements, add-ons, and deployment resources are hosted on GitHub repositories. Explore the following repositories to access the source code of components.
To maintain an active presence on social media channels and share valuable content about our software releases, we continuously publish materials across the following media:
The EPAM Delivery Platform (EDP) Developer Guide serves as a comprehensive technical resource specifically designed for developers. It offers detailed insights into expanding the functionalities of EDP. This section focuses on explaining the development approach and fundamental architectural blueprints that form the basis of the platform's ecosystem.
Within these pages, you'll find architectural diagrams, component schemas, and deployment strategies essential for grasping the structural elements of EDP. These technical illustrations serve as references, providing a detailed understanding of component interactions and deployment methodologies. Understanding the architecture of EDP and integrating third-party solutions into its established framework enables the creation of efficient, scalable, and customizable solutions within the EPAM Delivery Platform.
The diagram below illustrates how GitHub repositories and Docker registries are interconnected within the EDP ecosystem.
As a publicly available product, the EPAM Delivery Platform relies on various channels to share information, gather feedback, and distribute new releases effectively. This section outlines the diverse channels through which users can engage with our platform and stay informed about the latest developments and enhancements.
Our product is presented on AWS and Civo marketplaces. It's essential to ensure that the product information on these platforms is up-to-date and accurately reflects the latest version of our software:
Our platform components, optional enhancements, add-ons, and deployment resources are hosted on GitHub repositories. Explore the following repositories to access the source code of components.
To maintain an active presence on social media channels and share valuable content about our software releases, we continuously publish materials across the following media:
This section provides a comprehensive overview of the EDP deployment approach on a Kubernetes cluster. EDP is designed and functions based on a set of key guiding principles:
Operator Pattern Approach:Approach is used for deployment and configuration, ensuring that the platform aligns with Kubernetes native methodologies (see schema below).
Loosely Coupling: EDP comprises several loosely coupled operators responsible for different platform parts. These operators can be deployed independently, enabling the most straightforward platform customization and delivery approach.
Kubernetes Operator
The following deployment diagram illustrates the platform's core components, which provide the minimum functional capabilities required for the platform operation: build, push, deploy, and run applications. The platform relies on several mandatory dependencies:
Ingress: An ingress controller responsible for routing traffic to the platform.
Tekton Stack: Includes Tekton pipelines, triggers, dashboard, chains, etc.
ArgoCD: Responsible for GitOps deployment.
EPAM Delivery Platform Deployment Diagram
Codebase Operator: Responsible for managing git repositories, versioning, and branching. It also implements Jira integration controller.
CD Pipeline Operator: Manages Continuous Delivery (CD) pipelines and CD stages (which is an abstraction of Kubernetes Namespace). Operator acts as the bridge between the artifact and deployment tools, like Argo CD. It defines the CD pipeline structure, artifacts promotion logic and triggers the pipeline execution.
Tekton Pipelines: Manages Tekton pipelines and processes events (EventListener, Interceptor) from Version Control Systems. The pipelines are integrated with external tools like SonarQube, Nexus, etc.
EDP Portal: This is the User Interface (UI) component, built on top of Headlamp.
Business applications are deployed on the platform using the CD Pipeline Operator and Argo CD. By default, the CD Pipeline Operator uses Argo CD as a deployment tool. However, it can be replaced with any other tool, like FluxCD, Spinnaker, etc. The target environment for the application deployment is a Kubernetes cluster where EDP is deployed, but it can be any other Kubernetes cluster.
\ No newline at end of file
+ Kubernetes Deployment - EPAM Delivery Platform
This section provides a comprehensive overview of the EDP deployment approach on a Kubernetes cluster. EDP is designed and functions based on a set of key guiding principles:
Operator Pattern Approach:Approach is used for deployment and configuration, ensuring that the platform aligns with Kubernetes native methodologies (see schema below).
Loosely Coupling: EDP comprises several loosely coupled operators responsible for different platform parts. These operators can be deployed independently, enabling the most straightforward platform customization and delivery approach.
Kubernetes Operator
The following deployment diagram illustrates the platform's core components, which provide the minimum functional capabilities required for the platform operation: build, push, deploy, and run applications. The platform relies on several mandatory dependencies:
Ingress: An ingress controller responsible for routing traffic to the platform.
Tekton Stack: Includes Tekton pipelines, triggers, dashboard, chains, etc.
ArgoCD: Responsible for GitOps deployment.
EPAM Delivery Platform Deployment Diagram
Codebase Operator: Responsible for managing git repositories, versioning, and branching. It also implements Jira integration controller.
CD Pipeline Operator: Manages Continuous Delivery (CD) pipelines and CD stages (which is an abstraction of Kubernetes Namespace). Operator acts as the bridge between the artifact and deployment tools, like Argo CD. It defines the CD pipeline structure, artifacts promotion logic and triggers the pipeline execution.
Tekton Pipelines: Manages Tekton pipelines and processes events (EventListener, Interceptor) from Version Control Systems. The pipelines are integrated with external tools like SonarQube, Nexus, etc.
EDP Portal: This is the User Interface (UI) component, built on top of Headlamp.
Business applications are deployed on the platform using the CD Pipeline Operator and Argo CD. By default, the CD Pipeline Operator uses Argo CD as a deployment tool. However, it can be replaced with any other tool, like FluxCD, Spinnaker, etc. The target environment for the application deployment is a Kubernetes cluster where EDP is deployed, but it can be any other Kubernetes cluster.
\ No newline at end of file
diff --git a/developer-guide/local-development/index.html b/developer-guide/local-development/index.html
index d2fd36064..8bd780d51 100644
--- a/developer-guide/local-development/index.html
+++ b/developer-guide/local-development/index.html
@@ -1,4 +1,4 @@
- Operator Development - EPAM Delivery Platform
This page is intended for developers with the aim to share details on how to set up the local environment and start coding in Go language for EPAM Delivery Platform.
We recommend using GoLand and enabling the Kubernetes plugin. Before installing plugins, make sure to save your work because IDE may require restarting.
To set up the cloned operator, follow the three steps below:
Configure Go Build Option. Open folder in GoLand, click the button and select the Go Build option:
Add configuration
Fill in the variables in Configuration tab:
In the Files field, indicate the path to the main.go file;
In the Working directory field, indicate the path to the operator;
In the Environment field, specify the namespace to watch by setting WATCH_NAMESPACE variable. It should equal default but it can be any other if required by the cluster specifications.
In the Environment field, also specify the platform type by setting PLATFORM_TYPE. It should equal either kubernetes or openshift.
Build config
Check cluster connectivity and variables. Local development implies working within local Kubernetes clusters. Kind (Kubernetes in Docker) is recommended so set this or another environment first before running code.
Testing and linting must be used before every single commit with no exceptions. The instructions for the commands below are written here.
It is mandatory to run test and lint to make sure the code passes the tests and meets acceptance criteria. Most operators are covered by tests so just run them by issuing the commands "make test" and "make lint":
make test
+ Operator Development - EPAM Delivery Platform
This page is intended for developers with the aim to share details on how to set up the local environment and start coding in Go language for EPAM Delivery Platform.
We recommend using GoLand and enabling the Kubernetes plugin. Before installing plugins, make sure to save your work because IDE may require restarting.
To set up the cloned operator, follow the three steps below:
Configure Go Build Option. Open folder in GoLand, click the button and select the Go Build option:
Add configuration
Fill in the variables in Configuration tab:
In the Files field, indicate the path to the main.go file;
In the Working directory field, indicate the path to the operator;
In the Environment field, specify the namespace to watch by setting WATCH_NAMESPACE variable. It should equal default but it can be any other if required by the cluster specifications.
In the Environment field, also specify the platform type by setting PLATFORM_TYPE. It should equal either kubernetes or openshift.
Build config
Check cluster connectivity and variables. Local development implies working within local Kubernetes clusters. Kind (Kubernetes in Docker) is recommended so set this or another environment first before running code.
Testing and linting must be used before every single commit with no exceptions. The instructions for the commands below are written here.
It is mandatory to run test and lint to make sure the code passes the tests and meets acceptance criteria. Most operators are covered by tests so just run them by issuing the commands "make test" and "make lint":
make test
The command "make test" should give the output similar to the following:
"make test" command
make lint
The command "make lint" should give the output similar to the following:
"make lint" command
Observe Auto-Generated Docs, API and Manifests⚓︎
The commands below are especially essential when making changes to API. The code is unsatisfactory if these commands fail.
Generate documentation in the .MD file format so the developer can read it:
make api-docs
The command "make api-docs" should give the output similar to the following:
"make api-docs" command with the file contents
There are also manifests within the operator that generate zz_generated.deepcopy.go file in /api/v1 directory. This file is necessary for the platform to work but it's time-consuming to fill it by yourself so there is a mechanism that does it automatically. Update it using the following command and check if it looks properly:
make generate
diff --git a/developer-guide/mk-docs-development/index.html b/developer-guide/mk-docs-development/index.html
index b4d8b63af..f64fcbbcd 100644
--- a/developer-guide/mk-docs-development/index.html
+++ b/developer-guide/mk-docs-development/index.html
@@ -1,4 +1,4 @@
- Working With Documentation - EPAM Delivery Platform
This section defines necessary steps to start developing the EDP documentation in the MkDocs Framework. The framework presents a static site generator with documentation written in Markdown. All the docs are configured with a YAML configuration file.
This section defines necessary steps to start developing the EDP documentation in the MkDocs Framework. The framework presents a static site generator with documentation written in Markdown. All the docs are configured with a YAML configuration file.
The EPAM Delivery Platform’s (EDP) Reference Architecture serves as a blueprint for software delivery, outlining the best practices, tools, and technologies leveraged by the platform to ensure efficient and high-quality software development. It provides a comprehensive guide to navigate the complexities of software delivery, from code to deployment.
EDP operates on Kubernetes, a leading open-source system for automating deployment, scaling, and management of containerized applications. It consolidates a variety of open-source tools, ensuring a flexible and adaptable system that can seamlessly run on any public cloud or on-premises infrastructure. This versatility allows for a wide range of deployment options, catering to diverse business needs and operational requirements.
The EPAM Delivery Platform (EDP) is built on a set of key principles that guide its design and functionality:
Managed Infrastructure and Container Orchestration: EDP is based on a platform that leverages managed infrastructure and container orchestration, primarily through Kubernetes or OpenShift.
Security: EDP places a high emphasis on security, covering aspects such as authentication, authorization, and Single Sign-On (SSO) for platform services.
Development and Testing Toolset: EDP provides a comprehensive set of tools for development and testing, ensuring a robust and reliable software delivery process.
Well-Established Engineering Process: EDP reflects EPAM’s well-established engineering practices (EngX) in its CI/CD pipelines and delivery analytics.
Open-Source and Cloud-Agnostic: As an open-source, cloud-agnostic solution, EDP can be run on any preferred Kubernetes or OpenShift clusters.
DevSecOps Excellence: EDP empowers DevSecOps by making security a mandatory quality gate.
Automated Testing: EDP ensures seamless and predictable regression testing through automated test analysis.
EDP encompasses a comprehensive CI/CD ecosystem integrating essential tools such as the Tekton and Argo CD, augmented by additional functionalities. Within this robust framework, EDP seamlessly integrates SonarQube for continuous code quality assessment, enabling thorough analysis and ensuring adherence to coding standards. Additionally, incorporating Static Application Security Testing (SAST) toolset fortifies platform's security posture by proactively identifying vulnerabilities within the codebase. EDP leverages dedicated artifact storage solutions to manage and version application artifacts securely, ensuring streamlined deployment processes and traceability throughout the software development lifecycle. See the reference architecture diagram below:
EPAM Delivery Platform Reference Architecture
Developers access the platform by authenticating with their corporate credentials. The platform utilizes OpenID Connect (OIDC) for authentication and authorization across all tools and Kubernetes clusters. Using OIDC, EDP establishes a unified and secure authentication mechanism, ensuring seamless access control and user authentication for all integrated tools. This standardized approach upholds strict security protocols, ensuring consistency in authentication and authorization policies across the platform ecosystem. To integrate existing Identity Providers (IdPs), Keycloak serves as an identity broker on the platform. EDP offers the keycloak-operator to streamline Keycloak integration within the platform.
Developers engage with the platform via the EDP Portal, an intuitive interface offering a comprehensive overview of the platform’s capabilities. This centralized hub facilitates seamless navigation and access to various platform tools and components. Within the EDP Portal, developers can generate new components (codebases). The platform integrates with version control systems, optimizing source code management, fostering collaboration, and streamlining code review processes. To create new codebases, developers utilize Application Templates, ensuring a standardized approach to application development. The platform accommodates a range of application templates such as Java, Node.js, .NET, Python, and more. Additionally, developers can design custom templates via the EDP Marketplace to cater to their specific requirements.
Tekton is a potent, adaptable, and cloud-native framework designed for crafting CI/CD systems. It offers a collection of shared, reusable components that empower developers to construct, test, and deploy applications across various cloud providers or on-premises systems. As a foundational element within the EDP CI/CD ecosystem, Tekton seamlessly integrates with other tools and services, providing a robust and adaptable framework for constructing CI/CD pipelines. Tekton Pipelines allow developers to efficiently build, test, and deploy applications, while Tekton Triggers initiate pipelines based on specific events.
The codebase operator is a crucial part of the platform ecosystem. It manages codebases, their creation, deletion, and scaffolding, as well as their associated resources. It provides versioning, branching, and release capabilities and enables seamless integration with Git servers and Jira.
The platform has various cloud-agnostic tools that offer different functionalities, such as artifact storage, static security analysis, and code quality assessment. These tools are accessible through pipelines and codebase controllers. Additionally, the platform supports integration with managed services from cloud providers to deliver its core functionality—for instance, AWS Parameter Store stores secrets and AWS ECR - container images. AzureDevops Artifacts is an option to store artifacts leveraging Azure Cloud capabilities. SonarCloud, a cloud-based version of SonarQube, can be integrated to conduct static code analysis.
The CD Pipeline Operator oversees CD pipelines and their related resources. It offers a collection of shared, reusable components for constructing CD pipelines. Integrated with Tekton and Argo CD, the CD Pipeline Operator harnesses their capabilities, ensuring a robust and dependable software delivery process. With a Kubernetes API interface, the CD Pipeline Operator facilitates the management of CD pipelines. It enables artifact promotion logic and the triggering of CD pipelines based on specific events, further enhancing the efficiency and adaptability of the software delivery workflow.
Argo CD is a pivotal deployment tool adopted within the platform, embracing the GitOps delivery approach. It serves as the foundation for deploying both operational and business workloads. EDP recommends running a dedicated Argo CD instance to manage operational workloads, employing the Kubernetes add-ons approach for streamlined management.
Production workloads operate in isolation within dedicated Kubernetes clusters to uphold stringent standards and ensure the utmost security and resource allocation. This approach guarantees the highest isolation and operational integrity levels for critical production systems, aligning with industry best practices. EDP strongly recommends utilizing a pull model for production deployment. In this model, production deployment is initiated by the Argo CD instance explicitly deployed for the production environment.
The Platform is meticulously engineered to uphold best practices in workload distribution across various environments, including development, testing (manual/automation), user acceptance (UAT), staging, and production. While lower environments like development and testing may feasibly share clusters for workload efficiency, EDP strongly advocates and enforces the necessity of segregating production workloads into dedicated clusters. This segregation ensures the highest isolation, security, and resource allocation levels for mission-critical production systems, adhering to industry standards and ensuring optimal operational integrity.
EDP harnesses the robust capabilities of Kubernetes in conjunction with a suite of powerful tools tailored for monitoring, logging, and tracing. It integrates the Prometheus stack within ecosystem, leveraging its metrics collection, storage, and querying capabilities to enable comprehensive monitoring of system performance and health. EDP runs OpenSearch for centralized logging, enabling efficient log aggregation, analysis, and management across the platform. Incorporating OpenTelemetry enables standardized and seamless observability data collection, facilitating deep insights into platform behavior and performance. Additionally, it allows for connection with external aggregators and tools that support the OpenTelemetry protocol (OTLP).
Platform and Tools
EDP integrates with GitLab, GitHub, and Gerrit for version control. These systems are foundational components enabling efficient source code management, collaboration, and code review processes.
Platform ensures robust security measures by leveraging OpenID Connect (OIDC) for authentication and authorization across all platform tools and Kubernetes clusters. By employing OIDC, EDP establishes a unified and secure authentication mechanism, enabling seamless access control and user authentication for all tools integrated into the platform. This standardized approach ensures stringent security protocols, maintaining authentication consistency and authorization policies across the platform ecosystem.
\ No newline at end of file
+ Reference Architecture - EPAM Delivery Platform
The EPAM Delivery Platform’s (EDP) Reference Architecture serves as a blueprint for software delivery, outlining the best practices, tools, and technologies leveraged by the platform to ensure efficient and high-quality software development. It provides a comprehensive guide to navigate the complexities of software delivery, from code to deployment.
EDP operates on Kubernetes, a leading open-source system for automating deployment, scaling, and management of containerized applications. It consolidates a variety of open-source tools, ensuring a flexible and adaptable system that can seamlessly run on any public cloud or on-premises infrastructure. This versatility allows for a wide range of deployment options, catering to diverse business needs and operational requirements.
The EPAM Delivery Platform (EDP) is built on a set of key principles that guide its design and functionality:
Managed Infrastructure and Container Orchestration: EDP is based on a platform that leverages managed infrastructure and container orchestration, primarily through Kubernetes or OpenShift.
Security: EDP places a high emphasis on security, covering aspects such as authentication, authorization, and Single Sign-On (SSO) for platform services.
Development and Testing Toolset: EDP provides a comprehensive set of tools for development and testing, ensuring a robust and reliable software delivery process.
Well-Established Engineering Process: EDP reflects EPAM’s well-established engineering practices (EngX) in its CI/CD pipelines and delivery analytics.
Open-Source and Cloud-Agnostic: As an open-source, cloud-agnostic solution, EDP can be run on any preferred Kubernetes or OpenShift clusters.
DevSecOps Excellence: EDP empowers DevSecOps by making security a mandatory quality gate.
Automated Testing: EDP ensures seamless and predictable regression testing through automated test analysis.
EDP encompasses a comprehensive CI/CD ecosystem integrating essential tools such as the Tekton and Argo CD, augmented by additional functionalities. Within this robust framework, EDP seamlessly integrates SonarQube for continuous code quality assessment, enabling thorough analysis and ensuring adherence to coding standards. Additionally, incorporating Static Application Security Testing (SAST) toolset fortifies platform's security posture by proactively identifying vulnerabilities within the codebase. EDP leverages dedicated artifact storage solutions to manage and version application artifacts securely, ensuring streamlined deployment processes and traceability throughout the software development lifecycle. See the reference architecture diagram below:
EPAM Delivery Platform Reference Architecture
Developers access the platform by authenticating with their corporate credentials. The platform utilizes OpenID Connect (OIDC) for authentication and authorization across all tools and Kubernetes clusters. Using OIDC, EDP establishes a unified and secure authentication mechanism, ensuring seamless access control and user authentication for all integrated tools. This standardized approach upholds strict security protocols, ensuring consistency in authentication and authorization policies across the platform ecosystem. To integrate existing Identity Providers (IdPs), Keycloak serves as an identity broker on the platform. EDP offers the keycloak-operator to streamline Keycloak integration within the platform.
Developers engage with the platform via the EDP Portal, an intuitive interface offering a comprehensive overview of the platform’s capabilities. This centralized hub facilitates seamless navigation and access to various platform tools and components. Within the EDP Portal, developers can generate new components (codebases). The platform integrates with version control systems, optimizing source code management, fostering collaboration, and streamlining code review processes. To create new codebases, developers utilize Application Templates, ensuring a standardized approach to application development. The platform accommodates a range of application templates such as Java, Node.js, .NET, Python, and more. Additionally, developers can design custom templates via the EDP Marketplace to cater to their specific requirements.
Tekton is a potent, adaptable, and cloud-native framework designed for crafting CI/CD systems. It offers a collection of shared, reusable components that empower developers to construct, test, and deploy applications across various cloud providers or on-premises systems. As a foundational element within the EDP CI/CD ecosystem, Tekton seamlessly integrates with other tools and services, providing a robust and adaptable framework for constructing CI/CD pipelines. Tekton Pipelines allow developers to efficiently build, test, and deploy applications, while Tekton Triggers initiate pipelines based on specific events.
The codebase operator is a crucial part of the platform ecosystem. It manages codebases, their creation, deletion, and scaffolding, as well as their associated resources. It provides versioning, branching, and release capabilities and enables seamless integration with Git servers and Jira.
The platform has various cloud-agnostic tools that offer different functionalities, such as artifact storage, static security analysis, and code quality assessment. These tools are accessible through pipelines and codebase controllers. Additionally, the platform supports integration with managed services from cloud providers to deliver its core functionality—for instance, AWS Parameter Store stores secrets and AWS ECR - container images. AzureDevops Artifacts is an option to store artifacts leveraging Azure Cloud capabilities. SonarCloud, a cloud-based version of SonarQube, can be integrated to conduct static code analysis.
The CD Pipeline Operator oversees CD pipelines and their related resources. It offers a collection of shared, reusable components for constructing CD pipelines. Integrated with Tekton and Argo CD, the CD Pipeline Operator harnesses their capabilities, ensuring a robust and dependable software delivery process. With a Kubernetes API interface, the CD Pipeline Operator facilitates the management of CD pipelines. It enables artifact promotion logic and the triggering of CD pipelines based on specific events, further enhancing the efficiency and adaptability of the software delivery workflow.
Argo CD is a pivotal deployment tool adopted within the platform, embracing the GitOps delivery approach. It serves as the foundation for deploying both operational and business workloads. EDP recommends running a dedicated Argo CD instance to manage operational workloads, employing the Kubernetes add-ons approach for streamlined management.
Production workloads operate in isolation within dedicated Kubernetes clusters to uphold stringent standards and ensure the utmost security and resource allocation. This approach guarantees the highest isolation and operational integrity levels for critical production systems, aligning with industry best practices. EDP strongly recommends utilizing a pull model for production deployment. In this model, production deployment is initiated by the Argo CD instance explicitly deployed for the production environment.
The Platform is meticulously engineered to uphold best practices in workload distribution across various environments, including development, testing (manual/automation), user acceptance (UAT), staging, and production. While lower environments like development and testing may feasibly share clusters for workload efficiency, EDP strongly advocates and enforces the necessity of segregating production workloads into dedicated clusters. This segregation ensures the highest isolation, security, and resource allocation levels for mission-critical production systems, adhering to industry standards and ensuring optimal operational integrity.
EDP harnesses the robust capabilities of Kubernetes in conjunction with a suite of powerful tools tailored for monitoring, logging, and tracing. It integrates the Prometheus stack within ecosystem, leveraging its metrics collection, storage, and querying capabilities to enable comprehensive monitoring of system performance and health. EDP runs OpenSearch for centralized logging, enabling efficient log aggregation, analysis, and management across the platform. Incorporating OpenTelemetry enables standardized and seamless observability data collection, facilitating deep insights into platform behavior and performance. Additionally, it allows for connection with external aggregators and tools that support the OpenTelemetry protocol (OTLP).
Platform and Tools
EDP integrates with GitLab, GitHub, and Gerrit for version control. These systems are foundational components enabling efficient source code management, collaboration, and code review processes.
Platform ensures robust security measures by leveraging OpenID Connect (OIDC) for authentication and authorization across all platform tools and Kubernetes clusters. By employing OIDC, EDP establishes a unified and secure authentication mechanism, enabling seamless access control and user authentication for all tools integrated into the platform. This standardized approach ensures stringent security protocols, maintaining authentication consistency and authorization policies across the platform ecosystem.
\ No newline at end of file
diff --git a/developer-guide/reference-cicd-pipeline/index.html b/developer-guide/reference-cicd-pipeline/index.html
index eabc9205e..79a4b8550 100644
--- a/developer-guide/reference-cicd-pipeline/index.html
+++ b/developer-guide/reference-cicd-pipeline/index.html
@@ -1 +1 @@
- Reference CI/CD Pipeline - EPAM Delivery Platform
This document provides an in-depth overview of the Continuous Integration and Continuous Delivery (CI/CD) pipeline reference architecture implemented within the EPAM Delivery Platform (EDP). The pipeline is designed to facilitate efficient and automated software deployment across diverse environments, leveraging a suite of tools and methodologies for enhanced reliability, scalability, and security.
The CI/CD pipeline within EDP orchestrates the software delivery process, encompassing several sequential stages to ensure robustness and reliability.
EPAM Delivery Platform Reference CI/CD Pipeline
The CI/CD Pipeline follows a modular and scalable architecture that leverages various tools to ensure the reliability and efficiency of the software delivery process. The architecture can be divided into stages, each responsible for specific tasks. Explore the key components involved in the pipeline and their functionalities:
Source Code: The pipeline starts with the source code, representing the application's codebase. Developers commit their changes to the source code repository, triggering the pipeline.
Validate Commit Message: The commit message validation component checks the format and content of the commit message. It ensures the commit message follows the correct format and includes a valid Tracking Issue key. It helps maintain a standardized commit message format throughout the application development.
Build: The Build component compiles the source code, runs unit tests, and generates the application artifact. It consumes the artifact from the Artifact Repository (Nexus), ensuring consistent and reliable builds.
SAST with SonarQube: The Static Analysis Security Testing (SAST) component utilizes SonarQube to analyze the source code for potential security vulnerabilities, code smells, and quality issues. This step helps identify and address security or code quality issues early in development.
SCA: The Software Composition Analysis (SCA) component performs dependency analysis using cdxgen, Dependency-Track, Semgrep, and DefectDojo. It checks for known vulnerabilities or license compliance issues in the application's dependencies. By identifying and resolving these issues, it ensures the security and stability of the software.
Publish: The Publish component publishes the application artifact to the Artifact Repository. It posts Docker images to the Docker Registry and stores binary artifacts in the Nexus Repository. This process ensures that the artifacts are securely stored and easily accessed for future deployments.
Deploy: The Deploy component uses Argo CD or Tekton to deploy applications to target environments in Kubernetes, leveraging Helm charts to ensure seamless deployment. Deploy to Test/Quality Assurance/Performance Environments: The final stages of the pipeline involve deploying the application to different environments for testing and quality assurance purposes. The results of the tests are consolidated and reported to the Report Portal, facilitating efficient test reporting and analysis.
The overall architecture of the CI/CD Pipeline ensures a streamlined and automated software delivery process, from source code to deployment. It provides developers with the necessary tools and processes to ensure their applications' quality, security, and scalability. Furthermore, Tekton Chains enhances supply chain security by signing and generating in-toto metadata that verifies the integrity of artifacts and the CI/CD Pipeline.
Note
The tools mentioned in this document are just examples and can be replaced with other tools that offer similar functionality. For instance, instead of Harbor for the Docker Registry, it is possible to use AWS ECR. Consider using Azure Artifacts or JFrog Artifactory instead of Nexus for the artifact repository. Instead of setting up an self-managed instance of SonarQube, leverage SonarCloud, the cloud-based version of SonarQube, as an alternative. The CI/CD Pipeline architecture is flexible and adaptable, allowing the use of different tools based on specific project requirements and platform preferences.
\ No newline at end of file
+ Reference CI/CD Pipeline - EPAM Delivery Platform
This document provides an in-depth overview of the Continuous Integration and Continuous Delivery (CI/CD) pipeline reference architecture implemented within the EPAM Delivery Platform (EDP). The pipeline is designed to facilitate efficient and automated software deployment across diverse environments, leveraging a suite of tools and methodologies for enhanced reliability, scalability, and security.
The CI/CD pipeline within EDP orchestrates the software delivery process, encompassing several sequential stages to ensure robustness and reliability.
EPAM Delivery Platform Reference CI/CD Pipeline
The CI/CD Pipeline follows a modular and scalable architecture that leverages various tools to ensure the reliability and efficiency of the software delivery process. The architecture can be divided into stages, each responsible for specific tasks. Explore the key components involved in the pipeline and their functionalities:
Source Code: The pipeline starts with the source code, representing the application's codebase. Developers commit their changes to the source code repository, triggering the pipeline.
Validate Commit Message: The commit message validation component checks the format and content of the commit message. It ensures the commit message follows the correct format and includes a valid Tracking Issue key. It helps maintain a standardized commit message format throughout the application development.
Build: The Build component compiles the source code, runs unit tests, and generates the application artifact. It consumes the artifact from the Artifact Repository (Nexus), ensuring consistent and reliable builds.
SAST with SonarQube: The Static Analysis Security Testing (SAST) component utilizes SonarQube to analyze the source code for potential security vulnerabilities, code smells, and quality issues. This step helps identify and address security or code quality issues early in development.
SCA: The Software Composition Analysis (SCA) component performs dependency analysis using cdxgen, Dependency-Track, Semgrep, and DefectDojo. It checks for known vulnerabilities or license compliance issues in the application's dependencies. By identifying and resolving these issues, it ensures the security and stability of the software.
Publish: The Publish component publishes the application artifact to the Artifact Repository. It posts Docker images to the Docker Registry and stores binary artifacts in the Nexus Repository. This process ensures that the artifacts are securely stored and easily accessed for future deployments.
Deploy: The Deploy component uses Argo CD or Tekton to deploy applications to target environments in Kubernetes, leveraging Helm charts to ensure seamless deployment. Deploy to Test/Quality Assurance/Performance Environments: The final stages of the pipeline involve deploying the application to different environments for testing and quality assurance purposes. The results of the tests are consolidated and reported to the Report Portal, facilitating efficient test reporting and analysis.
The overall architecture of the CI/CD Pipeline ensures a streamlined and automated software delivery process, from source code to deployment. It provides developers with the necessary tools and processes to ensure their applications' quality, security, and scalability. Furthermore, Tekton Chains enhances supply chain security by signing and generating in-toto metadata that verifies the integrity of artifacts and the CI/CD Pipeline.
Note
The tools mentioned in this document are just examples and can be replaced with other tools that offer similar functionality. For instance, instead of Harbor for the Docker Registry, it is possible to use AWS ECR. Consider using Azure Artifacts or JFrog Artifactory instead of Nexus for the artifact repository. Instead of setting up an self-managed instance of SonarQube, leverage SonarCloud, the cloud-based version of SonarQube, as an alternative. The CI/CD Pipeline architecture is flexible and adaptable, allowing the use of different tools based on specific project requirements and platform preferences.
\ No newline at end of file
diff --git a/developer-guide/telemetry/index.html b/developer-guide/telemetry/index.html
index b30b79e52..0a2dd0c0a 100644
--- a/developer-guide/telemetry/index.html
+++ b/developer-guide/telemetry/index.html
@@ -1,4 +1,4 @@
- Telemetry - EPAM Delivery Platform
The codebase operator for the EPAM Delivery Platform gathers anonymous data through telemetry. This data provides a clear picture of how the platform is being used and empowers the development team to make informed decisions and strategic enhancements to meet evolving operational needs. The anonymous data collected also plays an essential role in adopting a Software Development Life Cycle (SDLC) process strategically.
The codebase-operator collects the following data:
The version of the platform
The number of codebases created and their parameters: language (for example, Java, NodeJS, etc.), framework (for example, FastAPI, Flask, etc.), build tool (for example, Maven, Gradle, etc.), strategy (for example, Clone, Create, Import), and Type (for example, library, application, etc.)
The number of CD pipelines created and their parameters: deployment type (for example, Auto, Manual), and the number of stages
The number of Git providers connected to the platform and their types (for example, GitHub, GitLab, Gerrit)
Where Jira is enabled or not
The type of the Docker registry connected to the platform (for example, Docker Hub, Harbor, ECR)
The codebase operator for the EPAM Delivery Platform gathers anonymous data through telemetry. This data provides a clear picture of how the platform is being used and empowers the development team to make informed decisions and strategic enhancements to meet evolving operational needs. The anonymous data collected also plays an essential role in adopting a Software Development Life Cycle (SDLC) process strategically.
The codebase-operator collects the following data:
The version of the platform
The number of codebases created and their parameters: language (for example, Java, NodeJS, etc.), framework (for example, FastAPI, Flask, etc.), build tool (for example, Maven, Gradle, etc.), strategy (for example, Clone, Create, Import), and Type (for example, library, application, etc.)
The number of CD pipelines created and their parameters: deployment type (for example, Auto, Manual), and the number of stages
The number of Git providers connected to the platform and their types (for example, GitHub, GitLab, Gerrit)
Where Jira is enabled or not
The type of the Docker registry connected to the platform (for example, Docker Hub, Harbor, ECR)
Consult EDP Glossary section for definitions mentioned on this page and EDP Toolset to have a full list of tools used with the Platform. The below table contains a full list of features provided by EDP.
Features
Description
Cloud Agnostic
EDP runs on Kubernetes cluster, so any Public Cloud Provider which provides Kubernetes can be used. Kubernetes clusters deployed on-premises work as well
CI/CD for Microservices
EDP is initially designed to support CI/CD for Microservices running as containerized applications inside Kubernetes Cluster. EDP also supports CI for: - Terraform Modules, - Open Policy Rules, - Workflows for Java (8,11,17), JavaScript (React, Vue, Angular, Express, Antora), C# (.NET 6.0), Python (FastAPI, Flask, 3.8), Go (Beego, Operator SDK)
Version Control System (VCS)
EDP installs Gerrit as a default Source Code Management (SCM) tool. EDP also supports GitHub and GitLab integration
Branching Strategy
EDP supports Trunk-based development as well as GitHub/GitLab flow. EDP creates two Pipelines per each codebase branch: Code Review and Build
Repository Structure
EDP provides separate Git repository per each Codebase and doesn't work with Monorepo. However, EDP does support customization and runs helm-lint, dockerfile-lint steps using Monorepo approach.
Artifacts Versioning
EDP supports two approaches for Artifacts versioning: - default (BRANCH-[TECH_STACK_VERSION]-BUILD_ID) - EDP (MAJOR.MINOR.PATCH-BUILD_ID), which is SemVer. Custom versioning can be created by implementing get-version stage
Application Library
EDP provides baseline codebase templates for Microservices, Libraries, within create strategy while onboarding new Codebase
Stages Library
Each EDP Pipeline consists of pre-defined steps (stages).
CI Pipelines
EDP provides CI Pipelines for first-class citizens: - Applications (Microservices) based on Java (8,11,17), JavaScript (React, Vue, Angular, Express, Antora, Next.js), C# (.NET 3.1, .NET 6.0), Python (FastAPI, Flask, 3.8), Go (Beego, Gin, Operator SDK), Helm (Charts, Pipeline) - Libraries based on Java (8,11,17), JavaScript (React, Vue, Angular, Express), Python (FastAPI, Flask, 3.8), Groovy Pipeline (Codenarc), HCL (Terraform), Rego (OPA), Container (Docker), Helm (Charts, Pipeline), C#(.NET 3.1, .NET 6.0) - Autotests based on Java8, Java11, Java17
CD Pipelines
EDP provides capabilities to design CD Pipelines (in EDP Portal UI) for Microservices and defines logic for artifacts flow (promotion) from env to env. Artifacts promotion is performed automatically (Autotests), manually (User Approval) or combining both approaches
Autotests
EDP provides CI pipeline for autotest implemented in Java. Autotests can be used as Quality Gates in CD Pipelines
Custom Pipeline Library
EDP can be extended by introducing Custom Pipeline Library
Dynamic Environments
Each EDP CD Pipeline creates/destroys environment upon user requests
\ No newline at end of file
+ Basic Concepts - EPAM Delivery Platform
Consult EDP Glossary section for definitions mentioned on this page and EDP Toolset to have a full list of tools used with the Platform. The below table contains a full list of features provided by EDP.
Features
Description
Cloud Agnostic
EDP runs on Kubernetes cluster, so any Public Cloud Provider which provides Kubernetes can be used. Kubernetes clusters deployed on-premises work as well
CI/CD for Microservices
EDP is initially designed to support CI/CD for Microservices running as containerized applications inside Kubernetes Cluster. EDP also supports CI for: - Terraform Modules, - Open Policy Rules, - Workflows for Java (8,11,17), JavaScript (React, Vue, Angular, Express, Antora), C# (.NET 6.0), Python (FastAPI, Flask, 3.8), Go (Beego, Operator SDK)
Version Control System (VCS)
EDP installs Gerrit as a default Source Code Management (SCM) tool. EDP also supports GitHub and GitLab integration
Branching Strategy
EDP supports Trunk-based development as well as GitHub/GitLab flow. EDP creates two Pipelines per each codebase branch: Code Review and Build
Repository Structure
EDP provides separate Git repository per each Codebase and doesn't work with Monorepo. However, EDP does support customization and runs helm-lint, dockerfile-lint steps using Monorepo approach.
Artifacts Versioning
EDP supports two approaches for Artifacts versioning: - default (BRANCH-[TECH_STACK_VERSION]-BUILD_ID) - EDP (MAJOR.MINOR.PATCH-BUILD_ID), which is SemVer. Custom versioning can be created by implementing get-version stage
Application Library
EDP provides baseline codebase templates for Microservices, Libraries, within create strategy while onboarding new Codebase
Stages Library
Each EDP Pipeline consists of pre-defined steps (stages).
CI Pipelines
EDP provides CI Pipelines for first-class citizens: - Applications (Microservices) based on Java (8,11,17), JavaScript (React, Vue, Angular, Express, Antora, Next.js), C# (.NET 3.1, .NET 6.0), Python (FastAPI, Flask, 3.8), Go (Beego, Gin, Operator SDK), Helm (Charts, Pipeline) - Libraries based on Java (8,11,17), JavaScript (React, Vue, Angular, Express), Python (FastAPI, Flask, 3.8), Groovy Pipeline (Codenarc), HCL (Terraform), Rego (OPA), Container (Docker), Helm (Charts, Pipeline), C#(.NET 3.1, .NET 6.0) - Autotests based on Java8, Java11, Java17
CD Pipelines
EDP provides capabilities to design CD Pipelines (in EDP Portal UI) for Microservices and defines logic for artifacts flow (promotion) from env to env. Artifacts promotion is performed automatically (Autotests), manually (User Approval) or combining both approaches
Autotests
EDP provides CI pipeline for autotest implemented in Java. Autotests can be used as Quality Gates in CD Pipelines
Custom Pipeline Library
EDP can be extended by introducing Custom Pipeline Library
Dynamic Environments
Each EDP CD Pipeline creates/destroys environment upon user requests
\ No newline at end of file
diff --git a/glossary/index.html b/glossary/index.html
index fd6e6cae5..916b285e2 100644
--- a/glossary/index.html
+++ b/glossary/index.html
@@ -1 +1 @@
- Glossary - EPAM Delivery Platform
Get familiar with the definitions and context for the most useful EDP terms presented in table below.
Terms
Details
EDP Component - an item used in CI/CD process
EDP Portal UI - an EDP component that helps to manage, set up, and control the business entities.
Artifactory - an EDP component that stores all the binary artifacts. NOTE: Nexus is used as a possible implementation of a repository.
CI/CD Server - an EDP component that launches pipelines that perform the build, QA, and deployment code logic. NOTE: Tekton is used as a possible implementation of a CI/CD server.
Code Review tool - an EDP component that collaborates with the changes in the codebase. NOTE: Gerrit is used as a possible implementation of a code review tool.
Identity Server - an authentication server providing a common way to verify requests to all of the applications. NOTE: Keycloak is used as a possible implementation of an identity server.
Security Realm Tenant - a realm in identity server (e.g Keycloak) where all users' accounts and their access permissions are managed. The realm is unique for the identity server instance.
Static Code Analyzer - an EDP component that inspects continuously a code quality before the necessary changes appear in a master branch. NOTE: SonarQube is used as a possible implementation of a static code analyzer.
VCS (Version Control System) - a replication of the Gerrit repository that displays all the changes made by developers. NOTE: GitHub and GitLab are used as the possible implementation of a repository with the version control system.
EDP Business Entity - a part of the CI/CD process (the integration, delivery, and deployment of any codebase changes)
Application - a codebase type that is built as the binary artifact and deployable unit with the code that is stored in VCS. As a result, the application becomes a container and can be deployed in an environment.
Autotests - a codebase type that inspects a product (e.g. an application set) on a stage. Autotests are not deployed to any container and launched from the respective code stage.
CD Pipeline (Continuous Delivery Pipeline) - an EDP business entity that describes the whole delivery process of the selected application set via the respective stages. The main idea of the CD pipeline is to promote the application version between the stages by applying the sequential verification (i.e. the second stage will be available if the verification on the first stage is successfully completed). NOTE: The CD pipeline can include the essential set of applications with its specific stages as well.
CD Pipeline Stage - an EDP business entity that is presented as the logical gate required for the application set inspection. Every stage has one OpenShift project where the selected application set is deployed. All stages are sequential and promote applications one-by-one.
Codebase - an EDP business entity that possesses a code.
Codebase Branch - an EDP business entity that represents a specific version in a Git branch. Every codebase branch has a Codebase Docker Stream entity.
Codebase Docker Stream - a deployable component that leads to the application build and displays that the last build was verified on the specific stage. Every CD pipeline stage accepts a set of Codebase Docker Streams (CDS) that are input and output. SAMPLE: if an application1 has a master branch, the input CDS will be named as [app name]-[pipeline name]-[stage name]-[master] and the output after the passing of the DEV stage will be as follows: [app name]-[pipeline name]-[stage name]-[dev]-[verified].
Git Server - a custom resource that is responsible for integration with Version Control System (VCS), whether it is GitHub, GitLab or Gerrit.
Infrastructure - a codebase type that is used to define and manage the underlying infrastructure of projects using the Infrastructure as Code (IaC) approach, ensuring consistency and reproducibility.
Library - a codebase type that is built as the binary artifact, i.e. it`s stored in the Artifactory and can be uploaded by other applications, autotests or libraries.
Quality Gate - an EDP business entity that represents the minimum acceptable results after the testing. Every stage has a quality gate that should be passed to promote the application. The stage quality gate can be a manual approve from a QA specialist OR a successful autotest launch.
Quality Gate Type - this value defines trigger type that promotes artifacts (images) to the next environment in CD Pipeline. There are manual and automatic types of quality gates. The manual type means that the promoting process should be confirmed in Tekton. The automatic type promotes the images automatically in case there are no errors in the Allure Report. NOTE: If any of the test types is not passed, the CD pipeline will fail.
Trigger Type - a value that defines a trigger type used for the CD pipeline triggering. There are manual and automatic types of triggering. The manual type means that the CD pipeline should be triggered manually. The automatic type triggers the CD pipeline automatically as soon as the Codebase Docker Stream was changed.
Automated Tests - different types of automated tests that can be run on the environment for a specific stage.
Build Pipeline - a Tekton pipeline that builds a corresponding codebase branch in the Codebase.
Build Stage - a stage that takes place after the code has been submitted/merged to the repository of the main branch (the pull request from the feature branch is merged to the main one, the Patch set is submitted in Gerrit).
Code Review Pipeline - a Tekton pipeline that inspects the code candidate in the Code Review tool.
Code Review Stage - a stage where code is reviewed before it goes to the main branch repository of the version control system (the commit to the feature branch is pushed, the Patch set is created in Gerrit).
Deploy Pipeline - a Tekton pipeline that is responsible for the CD Pipeline Stage deployment with the full set of applications and autotests.
Deployment Stage - a part of the Continuous Delivery where artifacts are being deployed to environments.
EDP CI Pipelines - an orchestrator for stages that is responsible for the common technical events, e.g. initialization, in Tekton pipeline.
Environment - a part of the stage where the built and packed into an image application are deployed for further testing. It`s possible to deploy several applications to several environments (Team and Integration environments) within one stage.
Team Environment - an environment type that can be deployed at any time by the manual trigger of the Deploy pipeline where team or developers can check out their applications. NOTE: The promotion from such kind of environment is prohibited and developed only for the local testing.
OpenShift / Kubernetes (K8S)
ConfigMap - a resource that stores configuration data and processes the strings that do not contain sensitive information.
Docker Container - is a lightweight, standalone, and executable package.
Docker Registry - a store for the Docker Container that is created for the application after the Build pipeline performance.
OpenShift Web Console - a web console that enables to view, manage, and change OpenShift / K8S resources using browser.
Operator Framework - a deployable unit in OpenShift that is responsible for one or a set of resources and performs its life circle (adding, displaying, and provisioning).
Path - a route component that helps to find a specified path (e.g. /api) at once and skip the other.
Pod - the smallest deployable unit of the large microservice application that is responsible for the application launch. The pod is presented as the one launched Docker container. When the Docker container is collected, it will be kept in Docker Registry and then saved as Pod in the OpenShift project. NOTE: The Deployment Config is responsible for the Pod push, restart, and stop processes.
PV (Persistent Volume) - a cluster resource that captures the details of the storage implementation and has an independent lifecycle of any individual pod.
PVC (Persistent Volume Claim) - a user request for storage that can request specific size and access mode. PV resources are consumed by PVCs.
Route - a resource in OpenShift that allows getting the external access to the pushed application.
Secret - an object that stores and manages all the sensitive information (e.g. passwords, tokens, and SSH keys).
Service - an external connection point with Pod that is responsible for the network. A specific Service is connected to a specific Pod using labels and redirects all the requests to Pod as well.
Site - a route component (link name) that is created from the indicated application name and applies automatically the project name and a wildcard DNS record.
\ No newline at end of file
+ Glossary - EPAM Delivery Platform
Get familiar with the definitions and context for the most useful EDP terms presented in table below.
Terms
Details
EDP Component - an item used in CI/CD process
EDP Portal UI - an EDP component that helps to manage, set up, and control the business entities.
Artifactory - an EDP component that stores all the binary artifacts. NOTE: Nexus is used as a possible implementation of a repository.
CI/CD Server - an EDP component that launches pipelines that perform the build, QA, and deployment code logic. NOTE: Tekton is used as a possible implementation of a CI/CD server.
Code Review tool - an EDP component that collaborates with the changes in the codebase. NOTE: Gerrit is used as a possible implementation of a code review tool.
Identity Server - an authentication server providing a common way to verify requests to all of the applications. NOTE: Keycloak is used as a possible implementation of an identity server.
Security Realm Tenant - a realm in identity server (e.g Keycloak) where all users' accounts and their access permissions are managed. The realm is unique for the identity server instance.
Static Code Analyzer - an EDP component that inspects continuously a code quality before the necessary changes appear in a master branch. NOTE: SonarQube is used as a possible implementation of a static code analyzer.
VCS (Version Control System) - a replication of the Gerrit repository that displays all the changes made by developers. NOTE: GitHub and GitLab are used as the possible implementation of a repository with the version control system.
EDP Business Entity - a part of the CI/CD process (the integration, delivery, and deployment of any codebase changes)
Application - a codebase type that is built as the binary artifact and deployable unit with the code that is stored in VCS. As a result, the application becomes a container and can be deployed in an environment.
Autotests - a codebase type that inspects a product (e.g. an application set) on a stage. Autotests are not deployed to any container and launched from the respective code stage.
CD Pipeline (Continuous Delivery Pipeline) - an EDP business entity that describes the whole delivery process of the selected application set via the respective stages. The main idea of the CD pipeline is to promote the application version between the stages by applying the sequential verification (i.e. the second stage will be available if the verification on the first stage is successfully completed). NOTE: The CD pipeline can include the essential set of applications with its specific stages as well.
CD Pipeline Stage - an EDP business entity that is presented as the logical gate required for the application set inspection. Every stage has one OpenShift project where the selected application set is deployed. All stages are sequential and promote applications one-by-one.
Codebase - an EDP business entity that possesses a code.
Codebase Branch - an EDP business entity that represents a specific version in a Git branch. Every codebase branch has a Codebase Docker Stream entity.
Codebase Docker Stream - a deployable component that leads to the application build and displays that the last build was verified on the specific stage. Every CD pipeline stage accepts a set of Codebase Docker Streams (CDS) that are input and output. SAMPLE: if an application1 has a master branch, the input CDS will be named as [app name]-[pipeline name]-[stage name]-[master] and the output after the passing of the DEV stage will be as follows: [app name]-[pipeline name]-[stage name]-[dev]-[verified].
Git Server - a custom resource that is responsible for integration with Version Control System (VCS), whether it is GitHub, GitLab or Gerrit.
Infrastructure - a codebase type that is used to define and manage the underlying infrastructure of projects using the Infrastructure as Code (IaC) approach, ensuring consistency and reproducibility.
Library - a codebase type that is built as the binary artifact, i.e. it`s stored in the Artifactory and can be uploaded by other applications, autotests or libraries.
Quality Gate - an EDP business entity that represents the minimum acceptable results after the testing. Every stage has a quality gate that should be passed to promote the application. The stage quality gate can be a manual approve from a QA specialist OR a successful autotest launch.
Quality Gate Type - this value defines trigger type that promotes artifacts (images) to the next environment in CD Pipeline. There are manual and automatic types of quality gates. The manual type means that the promoting process should be confirmed in Tekton. The automatic type promotes the images automatically in case there are no errors in the Allure Report. NOTE: If any of the test types is not passed, the CD pipeline will fail.
Trigger Type - a value that defines a trigger type used for the CD pipeline triggering. There are manual and automatic types of triggering. The manual type means that the CD pipeline should be triggered manually. The automatic type triggers the CD pipeline automatically as soon as the Codebase Docker Stream was changed.
Automated Tests - different types of automated tests that can be run on the environment for a specific stage.
Build Pipeline - a Tekton pipeline that builds a corresponding codebase branch in the Codebase.
Build Stage - a stage that takes place after the code has been submitted/merged to the repository of the main branch (the pull request from the feature branch is merged to the main one, the Patch set is submitted in Gerrit).
Code Review Pipeline - a Tekton pipeline that inspects the code candidate in the Code Review tool.
Code Review Stage - a stage where code is reviewed before it goes to the main branch repository of the version control system (the commit to the feature branch is pushed, the Patch set is created in Gerrit).
Deploy Pipeline - a Tekton pipeline that is responsible for the CD Pipeline Stage deployment with the full set of applications and autotests.
Deployment Stage - a part of the Continuous Delivery where artifacts are being deployed to environments.
EDP CI Pipelines - an orchestrator for stages that is responsible for the common technical events, e.g. initialization, in Tekton pipeline.
Environment - a part of the stage where the built and packed into an image application are deployed for further testing. It`s possible to deploy several applications to several environments (Team and Integration environments) within one stage.
Team Environment - an environment type that can be deployed at any time by the manual trigger of the Deploy pipeline where team or developers can check out their applications. NOTE: The promotion from such kind of environment is prohibited and developed only for the local testing.
OpenShift / Kubernetes (K8S)
ConfigMap - a resource that stores configuration data and processes the strings that do not contain sensitive information.
Docker Container - is a lightweight, standalone, and executable package.
Docker Registry - a store for the Docker Container that is created for the application after the Build pipeline performance.
OpenShift Web Console - a web console that enables to view, manage, and change OpenShift / K8S resources using browser.
Operator Framework - a deployable unit in OpenShift that is responsible for one or a set of resources and performs its life circle (adding, displaying, and provisioning).
Path - a route component that helps to find a specified path (e.g. /api) at once and skip the other.
Pod - the smallest deployable unit of the large microservice application that is responsible for the application launch. The pod is presented as the one launched Docker container. When the Docker container is collected, it will be kept in Docker Registry and then saved as Pod in the OpenShift project. NOTE: The Deployment Config is responsible for the Pod push, restart, and stop processes.
PV (Persistent Volume) - a cluster resource that captures the details of the storage implementation and has an independent lifecycle of any individual pod.
PVC (Persistent Volume Claim) - a user request for storage that can request specific size and access mode. PV resources are consumed by PVCs.
Route - a resource in OpenShift that allows getting the external access to the pushed application.
Secret - an object that stores and manages all the sensitive information (e.g. passwords, tokens, and SSH keys).
Service - an external connection point with Pod that is responsible for the network. A specific Service is connected to a specific Pod using labels and redirects all the requests to Pod as well.
Site - a route component (link name) that is created from the indicated application name and applies automatically the project name and a wildcard DNS record.
\ No newline at end of file
diff --git a/index.html b/index.html
index 20f8fd1ae..814dfcbcb 100644
--- a/index.html
+++ b/index.html
@@ -148,4 +148,4 @@
width: 33.3%;
}
}
-
Build your delivery rocket
Boost your delivery with the development culture based on the modern CI/CD stack, golden path and self-service capabilities of the EPAM Delivery Platform (EDP).
Boost your delivery with the development culture based on the modern CI/CD stack, golden path and self-service capabilities of the EPAM Delivery Platform (EDP).
\ No newline at end of file
diff --git a/operator-guide/add-ons-overview/index.html b/operator-guide/add-ons-overview/index.html
index a964e9d69..b58e06837 100644
--- a/operator-guide/add-ons-overview/index.html
+++ b/operator-guide/add-ons-overview/index.html
@@ -1,4 +1,4 @@
- Install via Add-Ons - EPAM Delivery Platform
EDP Add-Ons is basically a Kubernetes-based structure that enables users to quickly install additional components for the platform using Argo CD applications.
Add-Ons have been introduced into EDP starting from version 3.4.0. They empower users to seamlessly incorporate the platform with various additional components, such as SonarQube, Nexus, Keycloak, Jira, and more. This eliminates the need for manual installations, as outlined in the Install EDP page.
In a nutshell, Add-Ons represent separate Helm Charts that imply to be installed by one click using the Argo CD tool.
All the Add-Ons are stored in our public GitHub repository adhering to the GitOps approach. Apart from default Helm and Git files, it contains both custom resources called Applications for Argo CD and application source code. The repository follows the GitOps approach to enable Add-Ons with the capability to rollback changes when needed. The repository structure is the following:
├── CHANGELOG.md
+ Install via Add-Ons - EPAM Delivery Platform
EDP Add-Ons is basically a Kubernetes-based structure that enables users to quickly install additional components for the platform using Argo CD applications.
Add-Ons have been introduced into EDP starting from version 3.4.0. They empower users to seamlessly incorporate the platform with various additional components, such as SonarQube, Nexus, Keycloak, Jira, and more. This eliminates the need for manual installations, as outlined in the Install EDP page.
In a nutshell, Add-Ons represent separate Helm Charts that imply to be installed by one click using the Argo CD tool.
All the Add-Ons are stored in our public GitHub repository adhering to the GitOps approach. Apart from default Helm and Git files, it contains both custom resources called Applications for Argo CD and application source code. The repository follows the GitOps approach to enable Add-Ons with the capability to rollback changes when needed. The repository structure is the following:
This page serves as a brief overview of all the advanced components within EDP. While these third-party tools are not mandatory, they significantly enhance the platform's capabilities, enabling the creation of a robust CI/CD environment.
Although we have dedicated instructions for all of these third-party tools, for those who installed EDP using cluster add-ons, we recommend installing them via add-ons correspondingly.
This page serves as a brief overview of all the advanced components within EDP. While these third-party tools are not mandatory, they significantly enhance the platform's capabilities, enabling the creation of a robust CI/CD environment.
Although we have dedicated instructions for all of these third-party tools, for those who installed EDP using cluster add-ons, we recommend installing them via add-ons correspondingly.
\ No newline at end of file
diff --git a/operator-guide/argocd-integration/index.html b/operator-guide/argocd-integration/index.html
index e676d3c3b..b8b586232 100644
--- a/operator-guide/argocd-integration/index.html
+++ b/operator-guide/argocd-integration/index.html
@@ -1,4 +1,4 @@
- Argo CD Integration - EPAM Delivery Platform
KubeRocketCI uses Argo CD as a part of the Continues Delivery/Continues Deployment implementation. Argo CD follows the best GitOps practices, uses Kubernetes native approach for the Deployment Management, has rich UI and required RBAC capabilities.
Both approaches can be deployed with High Availability (HA) or Non High Availability (non HA) installation manifests.
KubeRocketCI uses the HA deployment with the cluster-admin permissions, to minimize cluster resources consumption by sharing single Argo CD instance across multiple EDP Tenants. Please follow the installation instructions to deploy Argo CD.
Argo CD is deployed in a separate argocd namespace.
Argo CD uses a cluster-admin role for managing cluster-scope resources.
The control-plane application is created using the App of Apps approach, and its code is managed by the control-plane members.
The control-plane is used to onboard new Argo CD Tenants (Argo CD Projects - AppProject).
The EDP Tenant Member manages Argo CD Applications using kind: Application in the edpTenant namespace.
The App Of Apps approach is used to manage the EDP Tenants. Inspect the edp-grub repository structure that is used to provide the EDP Tenants for the Argo CD Projects:
edp-grub
+ Argo CD Integration - EPAM Delivery Platform
KubeRocketCI uses Argo CD as a part of the Continues Delivery/Continues Deployment implementation. Argo CD follows the best GitOps practices, uses Kubernetes native approach for the Deployment Management, has rich UI and required RBAC capabilities.
Both approaches can be deployed with High Availability (HA) or Non High Availability (non HA) installation manifests.
KubeRocketCI uses the HA deployment with the cluster-admin permissions, to minimize cluster resources consumption by sharing single Argo CD instance across multiple EDP Tenants. Please follow the installation instructions to deploy Argo CD.
Argo CD is deployed in a separate argocd namespace.
Argo CD uses a cluster-admin role for managing cluster-scope resources.
The control-plane application is created using the App of Apps approach, and its code is managed by the control-plane members.
The control-plane is used to onboard new Argo CD Tenants (Argo CD Projects - AppProject).
The EDP Tenant Member manages Argo CD Applications using kind: Application in the edpTenant namespace.
The App Of Apps approach is used to manage the EDP Tenants. Inspect the edp-grub repository structure that is used to provide the EDP Tenants for the Argo CD Projects:
edp-grub
├──LICENSE
├──README.md
├──apps### All Argo CD Applications are stored here
diff --git a/operator-guide/artifacts-verification/index.html b/operator-guide/artifacts-verification/index.html
index d782ed0da..12f4d6911 100644
--- a/operator-guide/artifacts-verification/index.html
+++ b/operator-guide/artifacts-verification/index.html
@@ -1,4 +1,4 @@
- Verification of EDP Artifacts - EPAM Delivery Platform
This documentation outlines platform SLSA integration and guides verifying image authenticity and provenance.
Supply Chain Levels of Software Assurance (SLSA) is a framework for assessing and enhancing software supply chain security. Software Supply Chain Security is a critical aspect of modern software development and deployment. Supply Chain Levels of Software Assurance (SLSA) provides a framework for assessing and enhancing the security of your software supply chain.
EPAM Delivery Platform's container images are signed using cosign with the cosign.pub key for signing and transparency. You can verify a container image's signature by executing the cosign verify command.
To confirm the authenticity of the image, run the cosign verify command. See the example below:
cosignverify--keyhttps://raw.githubusercontent.com/epam/edp-install/master/cosign.pubepamedp/codebase-operator:2.20.0|jq.
+ Verification of EDP Artifacts - EPAM Delivery Platform
This documentation outlines platform SLSA integration and guides verifying image authenticity and provenance.
Supply Chain Levels of Software Assurance (SLSA) is a framework for assessing and enhancing software supply chain security. Software Supply Chain Security is a critical aspect of modern software development and deployment. Supply Chain Levels of Software Assurance (SLSA) provides a framework for assessing and enhancing the security of your software supply chain.
EPAM Delivery Platform's container images are signed using cosign with the cosign.pub key for signing and transparency. You can verify a container image's signature by executing the cosign verify command.
To confirm the authenticity of the image, run the cosign verify command. See the example below:
This documentation provides the detailed instructions on how to install the EPAM Delivery Platform via the AWS Marketplace.
To initiate the installation process, navigate to our dedicated AWS Marketplace page and commence the deployment of EPAM Delivery Platform.
Disclaimer
EDP is aligned with industry standards for storing and managing sensitive data, ensuring optimal security. However, the use of custom solutions introduces uncertainties, thus the responsibility for the safety of your data is totally covered by platform administrator.
A basic understanding of AWS services and navigation is preferred to facilitate smoother setup and deployment processes. If you are new to AWS, please refer to the AWS Documentation for detailed information on the services and their usage.
Understanding of Kubernetes: Knowledge of Kubernetes concepts and architecture is recommended for effective management and operation of clusters.
Please familiarize yourself with the Prerequisites page before deploying the product. To perform a minimal installation, ensure that you meet the following requirements:
The AWS Elastic Kubernetes Service (EKS) cluster is available for deployment. For detailed instructions on creating a new cluster, please consult the AWS EKS Cluster Creation Guide. Additionally, you can refer to our EKS Deployment Guide for step-by-step instructions tailored to your specific requirements.
The domain name is available and associated with the ingress object in cluster.
This documentation provides the detailed instructions on how to install the EPAM Delivery Platform via the AWS Marketplace.
To initiate the installation process, navigate to our dedicated AWS Marketplace page and commence the deployment of EPAM Delivery Platform.
Disclaimer
EDP is aligned with industry standards for storing and managing sensitive data, ensuring optimal security. However, the use of custom solutions introduces uncertainties, thus the responsibility for the safety of your data is totally covered by platform administrator.
A basic understanding of AWS services and navigation is preferred to facilitate smoother setup and deployment processes. If you are new to AWS, please refer to the AWS Documentation for detailed information on the services and their usage.
Understanding of Kubernetes: Knowledge of Kubernetes concepts and architecture is recommended for effective management and operation of clusters.
Please familiarize yourself with the Prerequisites page before deploying the product. To perform a minimal installation, ensure that you meet the following requirements:
The AWS Elastic Kubernetes Service (EKS) cluster is available for deployment. For detailed instructions on creating a new cluster, please consult the AWS EKS Cluster Creation Guide. Additionally, you can refer to our EKS Deployment Guide for step-by-step instructions tailored to your specific requirements.
The domain name is available and associated with the ingress object in cluster.
This article outlines how the EPAM Delivery Platform (EDP) leverages Capsule capabilities to enable isolation for both the core platform components and the business applications workload.
EPAM Delivery Platform uses Capsule to ensure resource isolation. It is crucial to define constraints through the Capsule tenant approach. This approach serves two primary objectives: limiting the resources allocated to the EDP components and regulating the resources utilized by each deployed environment.
To ensure isolation for the core platform components, create the edp namespace under the Capsule tenant. Use the template provided in this instruction to create the Capsule tenant specifically for the core components.
The following scheme outlines the general steps involved in configuring Capsule for seamless integration with EDP. This process ensures efficient resource isolation, allowing for the deployment and management of both EDP core platform components and business application workloads. Follow the sequential flow of the scheme to successfully integrate Capsule with the EPAM Delivery Platform:
This article outlines how the EPAM Delivery Platform (EDP) leverages Capsule capabilities to enable isolation for both the core platform components and the business applications workload.
EPAM Delivery Platform uses Capsule to ensure resource isolation. It is crucial to define constraints through the Capsule tenant approach. This approach serves two primary objectives: limiting the resources allocated to the EDP components and regulating the resources utilized by each deployed environment.
To ensure isolation for the core platform components, create the edp namespace under the Capsule tenant. Use the template provided in this instruction to create the Capsule tenant specifically for the core components.
The following scheme outlines the general steps involved in configuring Capsule for seamless integration with EDP. This process ensures efficient resource isolation, allowing for the deployment and management of both EDP core platform components and business application workloads. Follow the sequential flow of the scheme to successfully integrate Capsule with the EPAM Delivery Platform:
Capsule Installation - This initial step involves setting up Capsule on your environment.
Tenant Configuration - Once Capsule is installed, the next critical step is configuring the Capsule tenant. This involves defining specific parameters and constraints to regulate the allocation of resources to EDP components.
Impersonation - Impersonation plays a role in managing user identities and permissions within the Capsule environment. This step ensures secure and controlled access to resources.
Create EDP Namespace - The creation of a dedicated namespace under Capsule is crucial for isolating and managing the core components of EDP. This step establishes the environment where EDP will be deployed.
Deploy EDP - The final step involves deploying the EPAM Delivery Platform within the configured Capsule environment.
To follow the instruction, check the following prerequisites:
terraform 0.14.10
hashicorp/aws = 4.8.0
mrparkers/keycloak >= 3.0.0
hashicorp/kubernetes ~> 2.9.0
kubectl = 1.22
kubelogin >= v1.25.1
Ensure that Keycloak has network availability for AWS (not in a private network).
Note
To connect OIDC with a cluster, install and configure the kubelogin plugin. For Windows, it is recommended to download the kubelogin as a binary and add it to your PATH.
The solution includes three types of the resources - AWS (EKS), Keycloak, Kubernetes. The left part of Keycloak resources remain unchanged after creation, thus allowing us to associate a claim for a user group membership. Other resources can be created, deleted or changed if needed. The most crucial from Kubernetes permissions are Kubernetes RoleBindings and ClusterRoles/Roles. Roles present a set of permissions, in turn RoleBindings map Kubernetes Role to representative Keycloak groups, so a group member can have just appropriate permissions.
To follow the instruction, check the following prerequisites:
terraform 0.14.10
hashicorp/aws = 4.8.0
mrparkers/keycloak >= 3.0.0
hashicorp/kubernetes ~> 2.9.0
kubectl = 1.22
kubelogin >= v1.25.1
Ensure that Keycloak has network availability for AWS (not in a private network).
Note
To connect OIDC with a cluster, install and configure the kubelogin plugin. For Windows, it is recommended to download the kubelogin as a binary and add it to your PATH.
The solution includes three types of the resources - AWS (EKS), Keycloak, Kubernetes. The left part of Keycloak resources remain unchanged after creation, thus allowing us to associate a claim for a user group membership. Other resources can be created, deleted or changed if needed. The most crucial from Kubernetes permissions are Kubernetes RoleBindings and ClusterRoles/Roles. Roles present a set of permissions, in turn RoleBindings map Kubernetes Role to representative Keycloak groups, so a group member can have just appropriate permissions.
In dynamic projects, changes to the container registry may be necessary. This section provides instructions for switching the container registry.
Warning
Exercise caution: Removing registry settings may disrupt your CI/CD process. New components created after changing the registry, including Components and Environments, will seamlessly function. However, existing 'Components' require additional steps, as outlined below.
To remove container registry integration from the EDP, follow the steps below:
1. In the EDP Portal main menu, navigate to EDP -> Configuration -> Registry.
2. Click the Reset registry button, type the confirm word and then click Confirm:
Registry settings
Update Registry for the Existing Components and Environments⚓︎
The EPAM Delivery Platform uses CodebaseImageStream custom resource to define Container Registry settings for the codebases. To update the registry for the existing codebases, follow the steps below:
List all the existing CodebaseImageStream CR(s) and copy their <name> and <codebase name> fields:
button and select the 
