[EPIC] TestGround Infra + Running tests

[tty47]

Description 📜

Hello team,

We have achieved results under this branch:
https://github.com/celestiaorg/testground-infra/tree/jose/hackground-k8s-tf

If you have more in mind, please, feel free to add them to the list 📝

Thanks in advance!

---

Some interesting articles 🔍

review-147

---

Tasks 🔖

What is needed the start running tests against the new infra implementation

Tasks

Beta Give feedback

We're having trouble rendering your tasklist, it's most likely has one of the following formatting errors:

• indented tasks (i.e. nested tasklist; support for this is coming!)
• an empty task (i.e. - [ ] on a line by itself)
• any empty new lines (before/after the tasklist or in between tasks)
• duplicate issue/pr links
• a draft task exceeds 512 characters

Please check the tasklists documentation for more information. 🔗 Feedback Discussion

Thank you for participating in the Private Beta ❤️

1. TestGround Components #278
   automation kubernetes testground +3
   
2. telegraf-operator #290
   kubernetes testground +2
   
3. ~networking configuration in manual steps for instances~
4. ~add UUID to the IAM Service Role - use a variable to use it later in the cluster~
5. Component: Redis #274
   automation +1
   
6. Component: InfluxDB #279
   automation kubernetes testground +3
   
7. We use the Amazon CNI #277
   kubernetes testground +2
   
8. Component: Multus #275
   automation testground +2
   
9. Component: Weave #276
   automation kubernetes testground +3
   
10. Component: Flannel #280
    automation kubernetes testground +3
    
11. ~EKS limitations~ *we go with Kops for now, we can manage the CP (ControlPlane) by ourselves*
12. k8s: efs resources - tg-cluster #286
    kubernetes testground +2
    
13. influxdb deployment as a replica (not daemonset)
14. fix: issues with loki & grafana - CRDs missing #292
    bug fix +2
    
15. docs: TG-infra - Document the process #297
    documentation +1
    
16. fix: load balancer - tg-daemon - permisisons issues #294
    fix +1
    
17. fix: persistentVolumeClaimRetentionPolicy available since v1.23.x #296
    fix kubernetes +2
    
18. feat: add multi master to kops #299
    bug enhancement fix kubernetes +4
    
19. fix: networking issues #303
    +0
    
20. feat: check autoscaling #304
    automation infrastructure kubernetes +3
    
21. tg: k8s latest version possible #305
    automation enhancement +2
    

---

Repository 🗂️

---

Issues 👨‍🔧

Closes: celestiaorg/test-infra#147

---

PS: Please, feel free to update any part of the ticket, any suggestion/idea is always welcome 😊

---

Best 🚀

Jose Ramon Mañes

[tty47]

update:

Issues creating the cluster...

• EKS has some limitations, Alex talked with one of the previous maintainers and he gave us the "why", they decided to create the cluster with KOPS at the beginning... the CP (Control Plane) in EKS, is limited. With KOPS (or any other alternative), you can specify the CP resources/limits.
• Docker.sock: link
  • we have to find a workaround to start the tg-daemon.

[tty47]

update:

I'm able to execute tests against the new cluster 🚀

[tty47]

update:

just for the record, the cluster can be created and provisioned from scratch in about 12-15'.

atm, we are having some issues when executing tests, the pods are frozen in the status: waiting for networking...

[tty47]

last but not least, scaling nodes tests (cluster flexibility)🥲

[tty47]

as a note/reminder.

in the future, it would be very nice to have: hardening the cluster + securing the services

we don't want to keep the cluster open for everybody (or bots)

[tty47]

celestiaorg/testground-infra#2

[Bidon15]

can be closed once we merge celestiaorg/testground-infra#2 (review)