Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/infra proxy pipeline #6617

Closed
wants to merge 140 commits into from
Closed

Feature/infra proxy pipeline #6617

wants to merge 140 commits into from

Conversation

GorillaGigabytes
Copy link
Contributor

@GorillaGigabytes GorillaGigabytes commented Jan 25, 2022 •
edited
Loading

🔩 Description: What code changed, and why?

Create a Demo pipeline for extraction and parsing of zip file and saving all the required info into the db.

⛓️ Related Resources

Migration Data Service, upload of zip file

#6562
#6563

👍 Definition of Done

Gat a demo pipeline ready which communicates using channels, from extraction of a zip file to storage all the info to the database.

👟 How to Build and Test the Change

rebuild components/infra-proxy-service/

Only Unzip routine is working flawlessly, and rest of the function, routines needs to be updated with actual data to test and run.

✅ Checklist

All PRs must tick these:

With occasional exceptions, all PRs from Progress employees must tick these:

  • Is the code clear? (complicated code or lots of comments--subdivide and use well-named methods, meaningful variable names, etc.)
  • Consistency checked? (user notifications, user prompts, visual patterns, code patterns, variable names)
  • Repeated code blocks eliminated? (adapt and reuse existing components, blocks, functions, etc.)
  • Spelling, grammar, typos checked? (at a minimum use make spell in any component directory)
  • Code well-formatted? (indents, line breaks, etc. improve rather than hinder readability)

All PRs from Progress employees should tick these if appropriate:

  • Tests added/updated? (all new code needs new tests)
  • Docs added/updated? (all customer-facing changes)

Please add a note next to any checkbox above if you are NOT ticking it.

📷 Screenshots, if applicable

@GorillaGigabytes GorillaGigabytes added automate-infra-proxy-api using for automate infra proxy backend api's infra-user-mgnt labels Jan 25, 2022
@GorillaGigabytes GorillaGigabytes self-assigned this Jan 25, 2022
@GorillaGigabytes GorillaGigabytes force-pushed the feature/infra_proxy_pipeline branch from b6126a6 to b52bf0e Compare January 25, 2022 10:49
sonali523 and others added 23 commits January 27, 2022 12:34
@sonali523 @vinay033
Script added for table 'users' (#5722)
c3c282f 
* Script added for table 'users'

Signed-off-by: root <[email protected]>

* updated down script

Signed-off-by: root <[email protected]>

* updated up script

Signed-off-by: root <[email protected]>

* updated up script

Signed-off-by: root <[email protected]>
@iamazzeez @vinay033
iam added on roles tab
f5d2180 
Signed-off-by: Abdul-Az <[email protected]>
@vinay033
add fallback and bldr channel env vars to envrc and set back to dev w…
36bcc3d 
…hen doing builds (#5741)

Signed-off-by: Rick Marry <[email protected]>
@sonali523 @vinay033
Script added for table 'org_users' (#5738)
ed1da75 
Signed-off-by: root <[email protected]>
@iamazzeez @vinay033
added edit policy
452be0d 
Signed-off-by: Abdul-Az <[email protected]>
@sonali523 @vinay033
Storage functions for users table (#5731)
404b274 
* Storage functions added for users table

Signed-off-by: root <[email protected]>

* Updated with review comments

Signed-off-by: root <[email protected]>
@chaitali-mane @vinay033
Chaitali/add server slider (#5728)
9966918 
* converted server modal into slider

Signed-off-by: chaitali-mane <[email protected]>

* Updated test cases

Signed-off-by: chaitali-mane <[email protected]>
@iamazzeez @vinay033
environments create, delete policy added
6874604 
Signed-off-by: Abdul-Az <[email protected]>
@iamazzeez @vinay033
edit env policy added
c766927 
Signed-off-by: Abdul-Az <[email protected]>
@iamazzeez @vinay033
added data bags policy
50c0b36 
Signed-off-by: Abdul-Az <[email protected]>
@chaitali-mane @vinay033
Chaitali/add server slider (#5728)
f0e5902 
* converted server modal into slider

Signed-off-by: chaitali-mane <[email protected]>

* Updated test cases

Signed-off-by: chaitali-mane <[email protected]>
@iamazzeez @vinay033
ui three dots fix
417cf07 
Signed-off-by: Abdul-Az <[email protected]>
@iamazzeez @vinay033
reverted iam actions
9f592ec 
Signed-off-by: Abdul-Az <[email protected]>
@vinay033
Add changes for the create server slider (#5755)
3c79c3d 
* added some changes for the create server slider

Signed-off-by: Vinay Sharma <[email protected]>

* added some minor changes

Signed-off-by: Vinay Sharma <[email protected]>
@iamazzeez @vinay033
removed roles iam
ed49fb8 
Signed-off-by: Abdul-Az <[email protected]>
@iamazzeez @vinay033
environments authz reverted
8a0a0a9 
Signed-off-by: Abdul-Az <[email protected]>
@iamazzeez @vinay033
databags authz reverted
cbe2ad1 
Signed-off-by: Abdul-Az <[email protected]>
@sonali523 @vinay033
Added API to get server users list from backend DB (#5760)
69bdb37 
Signed-off-by: root <[email protected]>
@chaitali-mane @vinay033
Added users tab in server details page (#5802)
2aea513 
* Added Users tab UI

Signed-off-by: chaitali-mane <[email protected]>

* Added enitites api integration

Signed-off-by: chaitali-mane <[email protected]>

* minor changes

Signed-off-by: chaitali-mane <[email protected]>

* Updated minor change

Signed-off-by: chaitali-mane <[email protected]>
@himanshi-chhabra @vinay033
Added css change for notification overlap slider. (#5781)
8704e44 
Signed-off-by: Himanshi Chhabra <[email protected]>
@sonali523 @vinay033
Command to add the sample automate infra server users (#5786)
4c79374 
* Shell script added to add the sample infra server users

Signed-off-by: root <[email protected]>

* Minor changes added

Signed-off-by: root <[email protected]>
@kalroy @vinay033
Renaming the sql files and modifying to incorporate WebUI key
e34f440 
Signed-off-by: Kallol Roy <[email protected]>
@iamazzeez @vinay033
infra_server_actions tests
ac9165f 
Signed-off-by: Abdul-Az <[email protected]>
sonali523 and others added 19 commits January 27, 2022 12:34
@sonali523 @vinay033
Infra proxy API to get the organisation of chef server (#6363)
146e139 
* Get organisation of infra server from chef server API added

Signed-off-by: sonali wale <[email protected]>

* minor changes added

Signed-off-by: sonali wale <[email protected]>

* Test cases added

Signed-off-by: sonali wale <[email protected]>

* Changes added in test cases

Signed-off-by: sonali wale <[email protected]>

* Added some changes in test cases

Signed-off-by: sonali wale <[email protected]>

* Removed test cases to check the existing one

Signed-off-by: sonali wale <[email protected]>

* added test cases changes

Signed-off-by: sonali wale <[email protected]>

* incorporating review comments

Signed-off-by: sonali wale <[email protected]>

* Minor changes added in test cases

Signed-off-by: sonali wale <[email protected]>

* added some logs

Signed-off-by: sonali wale <[email protected]>

* changes added in test cases

Signed-off-by: sonali wale <[email protected]>

* test cases changes added

Signed-off-by: sonali wale <[email protected]>

* Fixes added for repo health status

Signed-off-by: sonali wale <[email protected]>

* Fix added for repo health failure

Signed-off-by: sonali wale <[email protected]>

* Changes added in bash script of add users to remove credential id from users table

Signed-off-by: sonali wale <[email protected]>

* Conflicts resolved

Signed-off-by: sonali wale <[email protected]>
@vinay033
Server details page - UI should have the option to update the WebUI k…
ae92fba 
…ey (#6440)

* added changes to update the webui key on server details page

Signed-off-by: Vinay Sharma <[email protected]>

* added some minor changes

Signed-off-by: Vinay Sharma <[email protected]>
@vinay033
Show the validation status of the WebUI key on the Chef Server Metada…
b1b2b5b 
…ta (#6435)

* changes for the validate webui key on server details page

Signed-off-by: Vinay Sharma <[email protected]>

* added chanegs for the validate webui key

Signed-off-by: Vinay Sharma <[email protected]>

* added changes for the specs

Signed-off-by: Vinay Sharma <[email protected]>

* added changes for the validate webui key error effect

Signed-off-by: Vinay Sharma <[email protected]>
@sonali523 @vinay033
API to update the web ui key (#6433)
9a613a7 
* Update web ui key API added

Signed-off-by: sonali wale <[email protected]>

* Removed extra files

Signed-off-by: sonali wale <[email protected]>

* Read secret changes added

Signed-off-by: sonali wale <[email protected]>

* Conflicts resolved

Signed-off-by: sonali wale <[email protected]>

* Test cases added

Signed-off-by: sonali wale <[email protected]>

* Commented validate web ui key test cases

Signed-off-by: sonali wale <[email protected]>

* Changes added in mock file

Signed-off-by: sonali wale <[email protected]>

* uncomment validate web ui key test cases

Signed-off-by: sonali wale <[email protected]>
@iamazzeez @vinay033
infra_viewer_actions added (#6436)
53c1033 
Signed-off-by: Abdul-Az <[email protected]>
@iamazzeez @vinay033
Abdul/infra editor actions (#6453)
6a3441f 
* editor actions added

Signed-off-by: Abdul-Az <[email protected]>

* infra_editor test cases

Signed-off-by: Abdul-Az <[email protected]>

* test cases fix

Signed-off-by: Abdul-Az <[email protected]>
@sonali523 @chaitali-mane @vinay033
Changes to use web ui key in all the Infra Proxy APIs for Cookbooks, …
c227718 
…Policy Files, Policy Groups, Clients (#6457)

* Changes added to use the web ui key

Signed-off-by: sonali wale <[email protected]>

* added minor UI change

Signed-off-by: Chaitali Mane <[email protected]>

Co-authored-by: Chaitali Mane <[email protected]>
@sonali523 @vinay033
Database script for migration tables (#6500)
5c7fdd6 
* Script added for migration tables

Signed-off-by: sonali wale <[email protected]>

* Insert scripts added for migration type and status

Signed-off-by: sonali wale <[email protected]>

* Server id column added in migration table

Signed-off-by: sonali wale <[email protected]>

* Index added on server_id column in migration table

Signed-off-by: sonali wale <[email protected]>
@iamazzeez @vinay033
Abdul/infra editor actions (#6508)
59aeff4 
* editor actions added

Signed-off-by: Abdul-Az <[email protected]>

* infra_editor test cases

Signed-off-by: Abdul-Az <[email protected]>

* test cases fix

Signed-off-by: Abdul-Az <[email protected]>

* infra editor actions update

Signed-off-by: Abdul-Az <[email protected]>

* editor role fix

Signed-off-by: Abdul-Az <[email protected]>

* test cases fix

Signed-off-by: Abdul-Az <[email protected]>
@iamazzeez @vinay033
Abdul/infra project owner (#6512)
be3dd15 
* project owner wip

Signed-off-by: Abdul-Az <[email protected]>

* project owner actions

Signed-off-by: Abdul-Az <[email protected]>

* fix

Signed-off-by: Abdul-Az <[email protected]>

* doc update

Signed-off-by: Abdul-Az <[email protected]>

* pipeline fix

Signed-off-by: Abdul-Az <[email protected]>
@chaitali-mane @kalroy @vinay033
Updated Infra Proxy cypress tests for web UI key (#6511)
56ef9ef 
* Updated cypress tests for web ui key

Signed-off-by: Chaitali Mane <[email protected]>

* Key in uppercase

Signed-off-by: Chaitali Mane <[email protected]>

* Update for minor

Signed-off-by: Chaitali Mane <[email protected]>

* removed unused lines

Signed-off-by: Chaitali Mane <[email protected]>

* removed error unit tests

Signed-off-by: Chaitali Mane <[email protected]>

* removed iam lint error

Signed-off-by: Chaitali Mane <[email protected]>

* Adding Cypress env variable for WebUI key

Signed-off-by: Kallol Roy <[email protected]>

* Updated variable

Signed-off-by: Chaitali Mane <[email protected]>

* removed replace

Signed-off-by: Chaitali Mane <[email protected]>

* Adding variable

Signed-off-by: Chaitali Mane <[email protected]>

* minor change

Signed-off-by: Chaitali Mane <[email protected]>

* removed hard coded values

Signed-off-by: Chaitali Mane <[email protected]>

* Removed hard coded values

Signed-off-by: Chaitali Mane <[email protected]>

* Updated variables

Signed-off-by: Chaitali Mane <[email protected]>

Co-authored-by: Kallol Roy <[email protected]>
@YashviJain01 @vinay033
New version of protoc and added migrations proto files (#6578)
4b6991b 
Signed-off-by: Yashvi Jain <[email protected]>
@chaitali-mane @iamazzeez @vinay033
Added sync organization and user button (#6592)
98639c7 
* updated docs

Signed-off-by: Abdul-Az <[email protected]>

* cypress error fix

Signed-off-by: Abdul-Az <[email protected]>

* lint fixes

Signed-off-by: Abdul-Az <[email protected]>

* updated docs

Signed-off-by: Abdul-Az <[email protected]>

* cypress error fix

Signed-off-by: Abdul-Az <[email protected]>

* Added button as per new design

Signed-off-by: Chaitali Mane <[email protected]>

* update change

Signed-off-by: Chaitali Mane <[email protected]>

Co-authored-by: Abdul-Az <[email protected]>
@YashviJain01 @vinay033
Creating a new package migrations (#6595)
80ff2ba 
Signed-off-by: Yashvi Jain <[email protected]>
@sonali523 @vinay033
Storage functions for the migration phases (#6529)
7a002d5 
* Get organisation API changes added

Signed-off-by: sonali wale <[email protected]>

* minor changes added

Signed-off-by: sonali wale <[email protected]>

* Remove command to create an orgs

Signed-off-by: sonali wale <[email protected]>

* Client Mock changes

Signed-off-by: Yashvi Jain <[email protected]>

* Storage function for all the phases of migration

Signed-off-by: sonali wale <[email protected]>

* Minor changes added

Signed-off-by: sonali wale <[email protected]>

* Incorporated review comments

Signed-off-by: sonali wale <[email protected]>

* Minor changes added

Signed-off-by: sonali wale <[email protected]>

* Changes for review comments

Signed-off-by: sonali wale <[email protected]>

* Comments removed

Signed-off-by: sonali wale <[email protected]>

Co-authored-by: Yashvi Jain <[email protected]>
@sonali523 @vinay033
Migration stage script and storage functions added (#6619)
20ccc72 
Signed-off-by: sonali wale <[email protected]>
@YashviJain01 @vinay033
Addition of migration id and migration status in get server api (#6613)
4dbdfa8 
* resolving conflicts and rebasing it

Signed-off-by: Yashvi Jain <[email protected]>

* Removing the loggers

Signed-off-by: Yashvi Jain <[email protected]>

* Query changes for lastes migration status

Signed-off-by: Yashvi Jain <[email protected]>

* changing the response varibale

Signed-off-by: Yashvi Jain <[email protected]>
@vinay033 @kalroy
Changes to audit the log for infra proxy (#6622)
c5978ad 
* added changes for infra proxy audit log

* added reflect codes

Signed-off-by: Kallol Roy <[email protected]>

* added changes for the audit logs for infra

Signed-off-by: Vinay Sharma <[email protected]>

* removed commented code

Signed-off-by: Vinay Sharma <[email protected]>

* added some minor changes

Signed-off-by: Vinay Sharma <[email protected]>

Co-authored-by: Kallol Roy <[email protected]>
@YashviJain01 @vinay033 @kalroy
Migration Data service creation and created a rpc method for upload z…
7fda7f0 
…ip file (#6612)

* Changes for stream api for upload

Signed-off-by: Yashvi Jain <[email protected]>

* changes

Signed-off-by: Yashvi Jain <[email protected]>

* added some changes for the protofile

Signed-off-by: Vinay Sharma <[email protected]>

* added function changes

* added changes for the proto

* added changes to devide the chunks in bytes

* added changes for the validation

Signed-off-by: Vinay Sharma <[email protected]>

* adding status in database

Signed-off-by: Yashvi Jain <[email protected]>

* adding the request for server id and filename

Signed-off-by: Yashvi Jain <[email protected]>

* Adding  loggers and saving the failed migration phase

Signed-off-by: Yashvi Jain <[email protected]>

* optimising the code

Signed-off-by: Yashvi Jain <[email protected]>

* Changes for content-type

Signed-off-by: Yashvi Jain <[email protected]>

* added changes for the validation

Signed-off-by: Vinay Sharma <[email protected]>

* Upload to the hab svc data folder

Signed-off-by: Kallol Roy <[email protected]>

* Enhance log and modify migration phases

Signed-off-by: Kallol Roy <[email protected]>

* Granularize the migration stages

Signed-off-by: Kallol Roy <[email protected]>

* Granularize the migration stages

Signed-off-by: Kallol Roy <[email protected]>

Co-authored-by: Vinay Sharma <[email protected]>
Co-authored-by: Kallol Roy <[email protected]>
@vinay033 vinay033 force-pushed the feature/infra_user_management branch from a3618d2 to 7fda7f0 Compare January 27, 2022 07:04
Pappu Kumar added 2 commits January 28, 2022 13:38
Gotten demo pipeline ready
804b88a 
Signed-off-by: Pappu Kumar <[email protected]>
Data pipeline architecture
c1ba310 
Signed-off-by: Pappu Kumar <[email protected]>
@GorillaGigabytes GorillaGigabytes force-pushed the feature/infra_proxy_pipeline branch from a4cda88 to c1ba310 Compare January 28, 2022 08:09
Pappu Kumar and others added 2 commits January 28, 2022 13:41
Removed a file
8d07dc1 
Signed-off-by: Pappu Kumar <[email protected]>
@kalroy
Brainstorming ideas
dceab3a 
Signed-off-by: Kallol Roy <[email protected]>
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@vinay033 vinay033 force-pushed the feature/infra_user_management branch from 19df40a to 15dafca Compare January 31, 2022 17:54
@GorillaGigabytes
Copy link
Contributor Author

I have raised a different PR hence closing this one

@GorillaGigabytes GorillaGigabytes deleted the feature/infra_proxy_pipeline branch February 16, 2022 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kalroy kalroy Awaiting requested review from kalroy

@vinay033 vinay033 Awaiting requested review from vinay033

@sonali523 sonali523 Awaiting requested review from sonali523

Assignees

@GorillaGigabytes GorillaGigabytes

Labels
automate-infra-proxy-api using for automate infra proxy backend api's infra-user-mgnt
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@GorillaGigabytes @sonali523 @iamazzeez @chaitali-mane @vinay033 @himanshi-chhabra @kalroy @YashviJain01