How to configured S3 connection to upload/download artifacts programmatically

[polyaxon-team]

We’re implementing a few things and I’ve got a quick question.
I see the example here https://github.com/polyaxon/polyaxon/blob/faec6649ed6a09ad29365f17795a404cc714c22e/site/integrations/data-on-s3.md but I don’t quite understand how to setup that S3Service(...) object. what would I pass in? a connection? how do I create the s3 connection object to pass in? I’ve got the connection defined in polyaxon’s config/polyaxonfile but I’m not sure what to create in python there.

[polyaxon-team]

We have to update that section, those services are deprecated, I would use the new implementation:

from polyaxon.fs.fs import get_fs_from_name

fs = get_fs_from_name("model-registry-s3")

This will return a fully resolved s3fs object. More information about how to use the fs object: https://s3fs.readthedocs.io/en/latest/#examples.

Note 1: The s3 rquirement is not installed by default, you wiil need pip install "polyaxon[s3]"

Note2: You will have to request the connection:

run:
  connections: ["model-registry-s3"]

Also by requesting the connection, the secret/config will be available in the container, so you can also use boto3 automatically if you do not like the to the s3fs implementation.

The docs for:

• GCS
• S3
• Azure