Enable user to specify proxy settings

[jebbens]

A customer needs to specify http_proxy and no_proxy settings for compute resources like Lambda functions, ECS containers, EC2 instances, etc. so that network traffic will be properly routed through the http proxy managed by their sponsor agency's cloud hosting division and they will be able to use LISA.

[estohlmann]

Hey @jebbens, thank you for bringing this to our attention! We will review this PFR for potential inclusion in our next release. I will keep you up to date here!

[jebbens]

I recommend allowing a user to provide values for two variables, https_proxy and no_proxy. The first can be defined as: "Optional URL, including port, of an https proxy, e.g. Squid". The second can be defined as "An optional comma-separated list of endpoints, hostnames, or IP addresses whose traffic should not flow through the HTTP proxy."

If these values are present, they should be added as environment variables for any Lambda functions using the Python runtime and are attached to a VPC as HTTPS_PROXY and NO_PROXY. I do not know how/if other Lambda runtimes address this.

EC2 instances must also be configured to use these values, if present. Here is some example code to configure EKS nodes, but this will need to be modified per instructions on using an http proxy with ECS nodes.

TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 3600")
REGION=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/region)
DOMAIN=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/services/domain)
PARTITION=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/services/partition)
INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)

PROXY_URL=${outbound_proxy_url}
if [[ -n "$PROXY_URL" ]]; then
  # https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-proxy.html
  export HTTPS_PROXY="$PROXY_URL" # enables calls to service APIs & IMDS

  EKS_IPv4_RANGE=$(aws eks describe-cluster --region $REGION --name ${cluster_name} --query 'cluster.kubernetesNetworkConfig.serviceIpv4Cidr')
  MAC=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/mac/)
  VPC_CIDR=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" "http://169.254.169.254/latest/meta-data/network/interfaces/macs/$MAC/vpc-ipv4-cidr-blocks" | xargs | tr ' ' ',')

  NO_PROXY_LIST=$EKS_IPv4_RANGE,$VPC_CIDR,localhost,127.0.0.1,169.254.169.254,.internal,.eks.$DOMAIN,${no_proxy_endpoints}
  NO_PROXY_LIST=$(sed "s/,$//" <<< "$NO_PROXY_LIST")

  # Set proxy for future processes
  cloud-init-per instance env_proxy_config cat <<EOF >> /etc/environment
http_proxy="$PROXY_URL"
https_proxy="$PROXY_URL"
no_proxy="$NO_PROXY_LIST"
HTTP_PROXY="$PROXY_URL"
HTTPS_PROXY="$PROXY_URL"
NO_PROXY="$NO_PROXY_LIST"
AWS_DEFAULT_REGION="$REGION"
EOF

  # Configure containerd for the proxy
  mkdir -p /etc/systemd/system/containerd.service.d
  cloud-init-per instance docker_proxy_config tee <<EOF /etc/systemd/system/containerd.service.d/http-proxy.conf >/dev/null
[Service]
EnvironmentFile=/etc/environment
EOF

  # Configure the sandbox-image for the proxy
  mkdir -p /etc/systemd/system/sandbox-image.service.d
  cloud-init-per instance docker_proxy_config tee <<EOF /etc/systemd/system/sandbox-image.service.d/http-proxy.conf >/dev/null
[Service]
EnvironmentFile=/etc/environment
EOF

  # Configure the kubelet for the proxy
  cloud-init-per instance kubelet_proxy_config tee <<EOF /etc/systemd/system/kubelet.service.d/proxy.conf >/dev/null
[Service]
EnvironmentFile=/etc/environment
EOF

  # https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-proxy-with-ssm-agent.html#ssm-agent-proxy-systemd
  mkdir /etc/systemd/system/amazon-ssm-agent.service.d
  cat <<EOF >> /etc/systemd/system/amazon-ssm-agent.service.d/override.conf
[Service]
EnvironmentFile=/etc/environment
EOF

  # Reload the daemon to reflect proxy configurations at launch of instance; restart ssm agent
  cloud-init-per instance reload_daemon systemctl daemon-reload
  cloud-init-per instance restart_ssm systemctl restart amazon-ssm-agent

fi