Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add new secrets manager implementation (Azure Key Vault) #6181

Open
NishantJoshi00 opened this issue Sep 25, 2024 · 14 comments · May be fixed by #7330
Open

[FEATURE] Add new secrets manager implementation (Azure Key Vault) #6181

NishantJoshi00 opened this issue Sep 25, 2024 · 14 comments · May be fixed by #7330
Assignees
Labels
hacktoberfest Issues that are up for grabs for Hacktoberfest participants

Comments

@NishantJoshi00
Copy link
Member

NishantJoshi00 commented Sep 25, 2024

📝 Feature Description
We need to expand our secrets manager support by integrating Azure Key Vault. Currently, our system supports three secrets manager interfaces: AWS KMS, Hashicorp Vault, and Plaintext. Adding Azure Key Vault will provide users with an additional option for securely storing sensitive configuration settings.

🔨 Possible Implementation

  • Research Azure Key Vault API and integration methods.
  • Develop a new secrets manager interface for Azure Key Vault, following the patterns established by existing integrations (AWS KMS, Hashicorp Vault).
  • Implement necessary authentication and access methods for Azure Key Vault.
  • Create functions to store, retrieve, and manage secrets using Azure Key Vault.
  • Ensure the new integration adheres to our existing security standards and practices.
  • Update configuration options to allow users to select Azure Key Vault as their secrets manager.
  • Develop comprehensive tests to verify the functionality and security of the Azure Key Vault integration.

🔖 Note:
The implementation should maintain consistency with our existing secrets manager interfaces while accommodating any Azure Key Vault-specific features or requirements. Ensure that the integration is scalable and performant, considering potential high-load scenarios.

Submission Process:

  • Ask the maintainers for assignment of the issue, you can request for assignment by commenting on the issue itself.
  • Once assigned, submit a pull request (PR).
  • Maintainers will review and provide feedback, if any.
  • Maintainers can unassign issues due to inactivity, read more here.
  • For this issue, please raise a PR to juspay/hyperswitch-card-vault.

Refer here for Terms and conditions for the contest.

@NishantJoshi00
Copy link
Member Author

Links to: juspay/hyperswitch-card-vault#114

@NishantJoshi00 NishantJoshi00 added the hacktoberfest Issues that are up for grabs for Hacktoberfest participants label Sep 25, 2024
@gorakhnathy7 gorakhnathy7 transferred this issue from juspay/hyperswitch-card-vault Oct 1, 2024
@gagandeepp
Copy link

interested please assign @NishantJoshi00

@gorakhnathy7
Copy link
Collaborator

Hey @gagandeepp Assigning this to you, thanks for your interest!

@gagandeepp gagandeepp removed their assignment Oct 5, 2024
@TheVidz
Copy link

TheVidz commented Oct 5, 2024

As this issue has been un-assigned, can this be assigned to me please?

@gorakhnathy7
Copy link
Collaborator

Sure @TheVidz Assigning this to you.
Thanks for your interest!

@gorakhnathy7
Copy link
Collaborator

Hey @TheVidz

Kindly let us know, if you're still working on the issue? If you have any questions or need assistance, feel free to reach out in the community.

@TheVidz
Copy link

TheVidz commented Oct 26, 2024

I'm sorry @gorakhnathy7 , I couldn't figure it out, you may assign the issue if someone else can..

@Rafee-Mohamed
Copy link

Rafee-Mohamed commented Feb 16, 2025

Hi @NishantJoshi00 and @gorakhnathy7 ,
I am interested in working on this issue and believe it is still relevant. Could you please assign it to me?
Looking forward to your response. Thank you.

Rafee-Mohamed added a commit to Rafee-Mohamed/hyperswitch that referenced this issue Feb 20, 2025
Implement Azure key vault support for secret management and encryption management in external_services.
This can be used to securely storing sensitive configuration settings.

Fixes juspay#6181
@Rafee-Mohamed
Copy link

Rafee-Mohamed commented Feb 20, 2025

Hi @NishantJoshi00 and @gorakhnathy7,

While implementing this feature, I encountered a version conflict with syn crate between the async-trait crate and the azure-sdk. The external_services crate (and other workspace dependencies) use async-trait, which requires syn version 2.0.77, while azure-sdk requires syn version 2.0.87. Resolving this issue would enable me to proceed with testing, formatting, linting, and additional requirements.

Issue Details

cargo check
Updating git repository https://github.com/Azure/azure-sdk-for-rust.git
Updating crates.io index
error: failed to select a version for syn.
… required by package typespec_macros v0.1.0 (https://github.com/Azure/azure-sdk-for-rust.git?branch=main#68f7fc8f)
… which satisfies git dependency typespec_macros of package typespec_client_core v0.1.0 (https://github.com/Azure/azure-sdk-for-rust.git?branch=main#68f7fc8f)
… which satisfies git dependency typespec_client_core of package azure_identity v0.22.0 (https://github.com/Azure/azure-sdk-for-rust.git?branch=main#68f7fc8f)
… which satisfies git dependency azure_identity of package external_services v0.1.0 (/Users/mohamedrafeek/open-source-projects/hyperswitch/crates/external_services)
… which satisfies path dependency external_services (locked to 0.1.0) of package drainer v0.1.0 (/Users/mohamedrafeek/open-source-projects/hyperswitch/crates/drainer)
versions that meet the requirements ^2.0.87 are: 2.0.98, 2.0.97, 2.0.96, 2.0.95, 2.0.94, 2.0.93, 2.0.92, 2.0.91, 2.0.90, 2.0.89, 2.0.88, 2.0.87

all possible versions conflict with previously selected packages.

previously selected package syn v2.0.77
… which satisfies dependency syn = “^2.0.46” (locked to 2.0.77) of package async-trait v0.1.82
… which satisfies dependency async-trait = “^0.1.79” (locked to 0.1.82) of package drainer v0.1.0 (/Users/mohamedrafeek/open-source-projects/hyperswitch/crates/drainer)

failed to select a version for syn which could resolve this conflict

Since syn is used in two different versions within the repository, a direct upgrade is ambiguous:

cargo update -p syn –precise 2.0.87
Blocking waiting for file lock on package cache
error: There are multiple syn packages in your project, and the specification syn is ambiguous.
Please re-run this command with one of the following specifications:
[email protected]
[email protected]

Proposed Solution

Updating syn for all crates that depend on version 2.0.77 by running:

cargo update -p [email protected] –precise 2.0.87

This updates the workspace syn dependency version from 2.0.77 to 2.0.87 and modifies the Cargo.lock file accordingly. It also updates several other dependencies that rely on this version of syn.

Request for Guidance

Would updating the syn crate in this manner be an acceptable approach to resolve the issue?
If there are any alternative or preferred solutions, please let me know. Additionally, I have raised a PR with the initial changes. You can check it for reference.

Thank you!

@SanchithHegde
Copy link
Member

Hey @Rafee-Mohamed, I think running this cargo update command is an acceptable solution to your problem:

cargo update --precise 2.0.87 --package [email protected]

Also, when pulling the Azure SDK crates, could you ensure that they're pulled from crates.io and are not added as git dependencies? I believe the team published the crates to crates.io a day ago or so.

@Rafee-Mohamed
Copy link

Rafee-Mohamed commented Feb 21, 2025

Hi @SanchithHegde,

I’ve updated the dependencies from Git to cartes.io. While making this change, I wanted to confirm whether we should commit the Cargo.lock file as well.

In general, for applications (as opposed to libraries), committing Cargo.lock ensures reproducible builds by locking dependency versions. However, I couldn’t find any specific guidance on whether this practice is being followed in this project. Could you please clarify the approach regarding this?

Thanks!

@SanchithHegde
Copy link
Member

I’ve updated the dependencies from Git to cartes.io. While making this change, I wanted to confirm whether we should commit the Cargo.lock file as well.

Yes, please commit the Cargo.lock file as well.

@Rafee-Mohamed
Copy link

Hi @SanchithHegde ,

While raising a PR, I ran the predefined checks such as cargo check --all-features. However, since the project includes both v1 and v2 features, enabling --all-features results in conflicts due to overlapping type definitions.

cargo check --all-features
    Checking common_utils v0.1.0 (hyperswitch/crates/common_utils)
error[E0428]: the name `Payment` is defined multiple times
  --> crates/common_utils/src/events.rs:22:5
   |
18 |     Payment {
   |     ------- previous definition of the type `Payment` here
...
22 |     Payment {
   |     ^^^^^^^ `Payment` redefined here

I could not find specific instructions in the project guidelines on handling this scenario. I also attempted to exclude v2 while running the checks, but the issue persisted.

Could you clarify the recommended approach for running predefined checks, such as testing and formatting, in cases where mutually exclusive features lead to conflicts? Any guidance would be greatly appreciated.

Thanks!

@SanchithHegde
Copy link
Member

@Rafee-Mohamed Please install just and run the following commands, they should handle excluding the mutually exclusive features:

  • just clippy: This should pass, and have no warnings or errors.
  • just clippy_v2: I don't think your changes specifically would affect our v2 compilation / builds, but you can run this as well, just to be sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hacktoberfest Issues that are up for grabs for Hacktoberfest participants
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants