Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

s3_lifecycle - Add size thresholds #861

Open
1 task done
mdavis-xyz opened this issue Jan 13, 2022 · 3 comments · May be fixed by #2205
Open
1 task done

s3_lifecycle - Add size thresholds #861

mdavis-xyz opened this issue Jan 13, 2022 · 3 comments · May be fixed by #2205
Labels
feature This issue/PR relates to a feature request good first issue waiting_on_contributor Needs help. Feel free to engage to get things unblocked

Comments

@mdavis-xyz
Copy link
Contributor

Summary

STANDARD_IA and GLACIER classes in S3 include an overhead for access history data. For small files the overhead can be larger than the file itself, so you end up paying more to store small files in these supposedly cheaper classes.

The solution is to add a minimum size filter to the lifecycle config. The API supports this. The s3_lifecycle ansible module does not yet support these. We should add a min size filter (and also max size filter).

Issue Type

Feature Idea

Component Name

s3_lifecycle

Additional Information

In the boto3 call, ObjectSizeGreaterThan and ObjectSizeLessThan are optional fields which go next to Prefix under Filter. They are the number of bytes (not kB).

I propose this usage:

- name: Configure a lifecycle rule on a bucket to expire (delete) items with a prefix of /logs/ after 30 days
  community.aws.s3_lifecycle:
    name: mybucket
    expiration_days: 30
    prefix: logs/
    object_size_greater_than: 123
    object_size_less_than: 123
    status: enabled
    state: present

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@markuman markuman added feature This issue/PR relates to a feature request good first issue labels Jan 13, 2022
@alinabuzachis
Copy link
Contributor

@mdavis-xyz Thank you for this feature idea. Would you be willing to work on it and open a pull request?

@mdavis-xyz
Copy link
Contributor Author

I just found out that Lifecycles actually already have a minimum size filter for these transitions.

link to docs

Objects smaller than 128 KB – For the following transitions, Amazon S3 does not transition objects that are smaller than 128 KB:

From the S3 Standard or S3 Standard-IA storage classes to S3 Intelligent-Tiering or S3 Glacier Instant Retrieval.

From the S3 Standard storage class to S3 Standard-IA or S3 One Zone-IA.

Therefore I don't need this new Ansible functionality for my use case. So I probably won't write this until much later this year when I get some spare time (and after writing other PRs I more urgently need.)

@abikouo abikouo added the waiting_on_contributor Needs help. Feel free to engage to get things unblocked label Jan 20, 2022
@tremble tremble changed the title Add size thresholds to s3_lifecycle s3_lifecycle - Add size thresholds Feb 6, 2023
@leo4ever leo4ever linked a pull request Dec 17, 2024 that will close this issue
Open
@leo4ever
Copy link

@alinabuzachis - I have raised PR #2205 to include minimum and maximum size thresholds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature This issue/PR relates to a feature request good first issue waiting_on_contributor Needs help. Feel free to engage to get things unblocked
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support min max object size in s3_lifecycle module leo4ever/community.aws
5 participants
@leo4ever @markuman @mdavis-xyz @alinabuzachis @abikouo