Added --storage.tsdb.retention.size to retain data for certain size.

Signed-off-by: Kushal Shukla <[email protected]>

prombench/docs/eks.md

@@ -107,8 +107,36 @@ Run Prombench tests in [Elastic Kubernetes Service (EKS)](https://aws.amazon.com
     ```bash
     make node_create
     ```
+3. When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.
 
-3. **Deploy the Kubernetes Objects**:
+In this setup, you have two choices:
+
+Here’s how each option works:
+- **Option 1: Download data from object storage**
+
+   To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name ```object-config.yml```  with the necessary credentials as per your object storage. This secret enables access to the stored data.
+> Note: Make sure this secret applied before ```3b_prometheus-test_deployment.yaml```
+
+- **Option 2: Skip downloading data**
+
+If you don’t Want to Download data create an empty secret like this -
+
+```yaml
+# Empty Secret to Skip Downloading Data
+apiVersion: v1
+kind: Secret
+metadata:
+  name: bucket-secret
+  namespace: prombench-{{ .PR_NUMBER }} 
+type: Opaque
+stringData:
+  object-config.yml: 
+ 
+Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (`--storage.tsdb.retention.size`). 
+
+> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results. 
+
+4. **Deploy the Kubernetes Objects**:
 
     ```bash
     make resource_apply

prombench/docs/gke.md

@@ -96,8 +96,36 @@ Run Prombench tests in [Google Kubernetes Engine (GKE)](https://cloud.google.com
     ```bash
     make node_create
     ```
+3. When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.
 
-3. **Deploy the Kubernetes Objects**:
+In this setup, you have two choices:
+
+Here’s how each option works:
+- **Option 1: Download data from object storage**
+
+   To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name ```object-config.yml```  with the necessary credentials as per your object storage. This secret enables access to the stored data.
+> Note: Make sure this secret applied before ```3b_prometheus-test_deployment.yaml```
+
+- **Option 2: Skip downloading data**
+
+If you don’t Want to Download data create an empty secret like this -
+
+```yaml
+# Empty Secret to Skip Downloading Data
+apiVersion: v1
+kind: Secret
+metadata:
+  name: bucket-secret
+  namespace: prombench-{{ .PR_NUMBER }} 
+type: Opaque
+stringData:
+  object-config.yml: 
+ 
+Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (`--storage.tsdb.retention.size`). 
+
+> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results. 
+
+4. **Deploy the Kubernetes Objects**:
 
     ```bash
     make resource_apply

prombench/docs/kind.md

@@ -140,6 +140,10 @@ metadata:
 type: Opaque
 stringData:
   object-config.yml: 
+ 
+Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (`--storage.tsdb.retention.size`). 
+
+> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results. 
 
 3. Deploy the Kubernetes objects:
    > **_Note:_** If you encounter a `too many files open` error caused by promtail, increase the default value of `/proc/sys/fs/inotify/max_user_instances` from 128 to 512:

prombench/manifests/prombench/benchmark/3b_prometheus-test_deployment.yaml

@@ -81,7 +81,8 @@ spec:
           "--web.external-url=http://{{ .DOMAIN_NAME }}/{{ .PR_NUMBER }}/prometheus-pr",
           "--storage.tsdb.path=/prometheus",
           "--config.file=/etc/prometheus/prometheus.yml",
-          "--log.level=debug"
+          "--log.level=debug",
+          "--storage.tsdb.retention.size=5GB"
         ]
         volumeMounts:
         - name: config-volume
@@ -199,7 +200,8 @@ spec:
           "--web.external-url=http://{{ .DOMAIN_NAME }}/{{ .PR_NUMBER }}/prometheus-release",
           "--storage.tsdb.path=/prometheus",
           "--config.file=/etc/prometheus/prometheus.yml",
-          "--log.level=debug"
+          "--log.level=debug",
+          "--storage.tsdb.retention.size=5GB"
         ]
         volumeMounts:
         - name: config-volume