Refactor: Reduce Environment Variable Duplication

[CodeWithEmad]

Current Situation

Environment variables are currently duplicated across multiple files and deployment types:

• Kubernetes manifests (deployments and jobs)
• Docker Compose files (production, development, and jobs)
• Multiple service definitions (Superset, Clickhouse, Aspects, Ralph)

For example, Superset's environment variables are duplicated in:

• K8s deployment
• K8s worker deployment
• K8s jobs
• Docker Compose base service
• Docker Compose development overrides

Problems

1. Maintenance Overhead: Changes require updates in multiple places
2. Risk of Inconsistency: Different environments might drift apart if updates are missed
3. Poor DRY Practice: Same configuration repeated across different files
4. Mixed Approaches: Some services use env_files, others use direct environment variables

Proposed Solution

1. Create a centralized environment configuration structure.
2. Use Kubernetes ConfigMaps and Docker Compose env_files to load these configurations
3. Implement a clear override hierarchy for environment-specific settings

Benefits

• Single source of truth for configurations
• Easier maintenance and updates
• Consistent configuration across environments
• Clear separation between base and environment-specific settings

Related Files

• tutoraspects/patches/k8s-deployments
• tutoraspects/patches/k8s-jobs
• tutoraspects/templates/base-docker-compose-services
• tutoraspects/patches/local-docker-compose-*

[CodeWithEmad]

I'll fix this after #1106 was merged.