Skip to content

Conversation

@davidparks21
Copy link

@davidparks21 davidparks21 commented Jul 22, 2019

I've added 2 files under the k8s secrets named prp-s3-credentials, /root/.aws/credentials and /root/.s3cfg (for s3cmd, which I summarily declare trounces awscli for usability).

I've updated the pod.yml example here as follows:

  • The volumeMounts now mount only the file, not the entire folder (the mount is read-only, so that's useful).
  • I absolutely positively cannot find a way to make awscli respect a non AWS endpoint without a complex install of the endpoint plugin which was beyond what we should be doing in the secrets file. See: Ability to specify endpoint-url in profile aws/aws-cli#1270. So I just configured nautilus S3 as default and removed profile. Note: If someone needs to access AWS s3 they will need to modify these files anyway which they can't because they're read only under secrets, so some other solution will be necessary anyway.
  • Updated the endpoint in the k8s cluster to the high speed internal endpoint.
  • Added the .s3cfg config for s3cmd which contains the credentials as well. The prp-s3-credentials secret contains two files now.
  • Tested both awscli and s3cmd work in a pod as seen below.
root@davidparks21-pod:/app# aws --endpoint $AWS_S3_ENDPOINT s3 ls s3://braingeneers/
                           PRE archive/
                           PRE fashion-mnist/
                           PRE mea/
                           PRE nrezaee/
                           PRE rcurrie/
                           PRE simulated/
root@davidparks21-pod:/app# s3cmd ls s3://braingeneers/
                       DIR   s3://braingeneers/archive/
                       DIR   s3://braingeneers/fashion-mnist/
                       DIR   s3://braingeneers/mea/
                       DIR   s3://braingeneers/nrezaee/
                       DIR   s3://braingeneers/rcurrie/
                       DIR   s3://braingeneers/simulated/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant