Support encryption on object storage

[udf2457]

Is there a technical reason for the omission of encryption on object storage (S3) ?

This could be done even without introducing any extra code to parseable if you exposed support for S3-side features such as SSE-C.

[nitisht]

Can you elaborate a bit.

if you exposed support for S3-side features such as SSE-C.

What do we have to expose here?

[udf2457]

What do we have to expose here?

Basically all you need to do is introduce the extra env-vars and push that down to your S3 code so the correct HTTP headers are set.

For example, enabling SSE-C is as simple as sending the following headers when interacting with S3:

x-amz-server-side​-encryption​-customer-algorithm: currently a fixed static value, AES256
x-amz-server-side​-encryption​-customer-key: 256-bit, base64-encoded encryption key
x-amz-server-side​-encryption​-customer-key-MD5: base64-encoded 128-bit MD5 digest of the encryption key

S3 then does the crypto transparently in-line.

[nitisht]

I see, this is useful @udf2457 . We'll take a look soon.

[udf2457]

No problem.

People who use the AWS native S3 might point out they can (also) configure encryption via AWS S3 management (e.g. if you are 100% AWS-ecosystem you can configure so a bucket uses AWS KMS hosted keys).

So this feature is perhaps of most significant interest to those (like me 😉 ) who use S3-compatible services from other non-AWS providers or software (e.g. minio).

[nitisht]

/bounty 100

[algora-pbc]

💎 $100 bounty • Parseable

Steps to solve:

1. Start working: Comment /attempt #919 with your implementation plan
2. Submit work: Create a pull request including /claim #919 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to parseablehq/parseable!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @lazyfuhrer	Sep 16, 2024, 3:20:48 PM	WIP

[lazyfuhrer]

I want to work on this. Can I be assigned ? @nitisht

[lazyfuhrer]

/attempt #919

Algora profile	Completed bounties	Tech	Active attempts	Options
@lazyfuhrer	2 bounties from 1 project	TypeScript, JavaScript, Rust & more		Cancel attempt

[nitisht]

Sure @lazyfuhrer please go ahead.

[nitisht]

any updates @lazyfuhrer ?

[MihirLuthra]

Since this issue hasn't received any updates, I took this up today and have raised a PR.

I won't be taking the bounty though (may have some legal conflicts in my case)