Test merging strategy #39
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feat: add init project structure * feat(identity): scaffold db for default identity * docs(auth): add architecture diagram * feat(config): add jwt config * perf: move endpoints to separate proj than di * feat: add migrations for default identity * feat(infr): add new migrations for custom user * feat(domain): add user entity and jwt config * perf(di): add custom user to identity auth * feat(test): scaffold integration tests * feat(app): implement handlers required by auth -CreateUser -ValidateUserCredentials -GenerateJwtToken * refactor(test): split test proj between integration and unit * perf(mapper): use automapper for register endpoint * perf(app): organize handlers based on vertical slice * feat(test): add fixture for test containers * feat(test): create integration test for register endpoint * perf(api): refactor endpoints to improve readability * fix(test): remove sqlconnection error and speed up test containers - Declare additional IServiceCollection extension methods for ensuring db created and removing already existing DbContextOptions<AppDbContext> (not AppDbContext!!) - Use the latest MsSql Docker Image * feat(di): add default swagger support * test: remove unit tests proj As Fastendpoints Commands are hard to unit test outside of an Endpoint, it is enough to have only Integration tests to cover everything * test(login): add integration test for login * style: rename username prop and move jwt mapper here * docs(swagger): add swagger support * feat(users): scaffold CRUD users api * style(auth): move validators to separate files * feat(users): add get all endpoint * test(users): add integration tests for get all endpoint * feat(users): add get user endpoint * test(users): add integration test for get user endpoint * style(tests): add regions to enhance readability * style: replace local usings with global usings * perf(user): add guid as pk for application user * feat(update): create update user endpoint * test(update): create integration tests for update endpoint * misc: update .gitignore * perf(user manager): use predefined methods instead of LINQ for finding by id * feat(api): create delete endpoint * style: add regions and update global usings * test(delete): create integration tests for delete endpoint * docs(swagger): add summaries for Identity CRUD endpoints * style(binding): remove redundant bind from attribute for models Id * perf(update); enable request fields to be nullable In order to allow partial update for user profile (similar to patch), enable validation/mapping rules only when a certain request field is provided * style: add global usings * test(update): add integration test for partial update * style: format code * style: adjust namespace for jwt config * feat(user): add support for fullname and bio fields * test(user): update integration tests to support fullname/bio fields * feat(image): add support for user profile image * test(register): update integration tests to support profile image * perf(picture): enable support for picture in login endpoint Enable eager loading in db context for profile picture in order to be able to retrieve the image id for the download URL * feat(build): add migrations for profile picture * test(login): update login tests for picture Replace register endpoint invocation with the user manager directly in order to mitigate the tight coupling between tests and the QAPI * feat(picture): update the rest of endpoints to support profile picture * test(picture): update the rest of tests to support profile picture * feat(image): add get image endpoint * test(image): add integration test for get image endpoint * perf(image): create image repository * style: format code * perf: add argument null guards to ctors * feat(logger): configure serilog * misc: add Logs to .gitignore * perf: add logging to endpoints * fix(logger): replace bootstrap logger with default Bug related to integration tests fails * feat(update): add permission guard to endpoint Allow users to edit only their own profile, otherwise throw unauthorized * test: add integration test for self update permission * perf(update): replace unauthorized with forbidden * perf: use default identity role * perf: set model fields as init-only * feat: add seed data for roles and admin user * test(get all): remove users already added by seeding * perf(delete): restrict access to endpoint using roles from jwt * test(delete): add test for non-admin user * perf(image): enable profile image to be nullable * feat(api): add support for refresh tokens Add RefreshToken endpoint and pass access/refresh token pairs via HttpOnly cookies * perf(update user): get user id exclusively from route * docs: add sequence flow diagrams for different scenarios
* feat: add init project structure * feat(identity): scaffold db for default identity * docs(auth): add architecture diagram * feat(config): add jwt config * perf: move endpoints to separate proj than di * feat: add migrations for default identity * feat(infr): add new migrations for custom user * feat(domain): add user entity and jwt config * perf(di): add custom user to identity auth * feat(test): scaffold integration tests * feat(app): implement handlers required by auth -CreateUser -ValidateUserCredentials -GenerateJwtToken * refactor(test): split test proj between integration and unit * perf(mapper): use automapper for register endpoint * perf(app): organize handlers based on vertical slice * feat(test): add fixture for test containers * feat(test): create integration test for register endpoint * perf(api): refactor endpoints to improve readability * fix(test): remove sqlconnection error and speed up test containers - Declare additional IServiceCollection extension methods for ensuring db created and removing already existing DbContextOptions<AppDbContext> (not AppDbContext!!) - Use the latest MsSql Docker Image * feat(di): add default swagger support * test: remove unit tests proj As Fastendpoints Commands are hard to unit test outside of an Endpoint, it is enough to have only Integration tests to cover everything * test(login): add integration test for login * style: rename username prop and move jwt mapper here * docs(swagger): add swagger support * feat(users): scaffold CRUD users api * style(auth): move validators to separate files * feat(users): add get all endpoint * test(users): add integration tests for get all endpoint * feat(users): add get user endpoint * test(users): add integration test for get user endpoint * style(tests): add regions to enhance readability * style: replace local usings with global usings * perf(user): add guid as pk for application user * feat(update): create update user endpoint * test(update): create integration tests for update endpoint * misc: update .gitignore * perf(user manager): use predefined methods instead of LINQ for finding by id * feat(api): create delete endpoint * style: add regions and update global usings * test(delete): create integration tests for delete endpoint * docs(swagger): add summaries for Identity CRUD endpoints * style(binding): remove redundant bind from attribute for models Id * perf(update); enable request fields to be nullable In order to allow partial update for user profile (similar to patch), enable validation/mapping rules only when a certain request field is provided * style: add global usings * test(update): add integration test for partial update * style: format code * style: adjust namespace for jwt config * feat(user): add support for fullname and bio fields * test(user): update integration tests to support fullname/bio fields * feat(image): add support for user profile image * test(register): update integration tests to support profile image * perf(picture): enable support for picture in login endpoint Enable eager loading in db context for profile picture in order to be able to retrieve the image id for the download URL * feat(build): add migrations for profile picture * test(login): update login tests for picture Replace register endpoint invocation with the user manager directly in order to mitigate the tight coupling between tests and the QAPI * feat(picture): update the rest of endpoints to support profile picture * test(picture): update the rest of tests to support profile picture * feat(image): add get image endpoint * test(image): add integration test for get image endpoint * perf(image): create image repository * style: format code * perf: add argument null guards to ctors * feat(logger): configure serilog * misc: add Logs to .gitignore * perf: add logging to endpoints * fix(logger): replace bootstrap logger with default Bug related to integration tests fails * feat(update): add permission guard to endpoint Allow users to edit only their own profile, otherwise throw unauthorized * test: add integration test for self update permission * perf(update): replace unauthorized with forbidden * perf: use default identity role * perf: set model fields as init-only * feat: add seed data for roles and admin user * test(get all): remove users already added by seeding * perf(delete): restrict access to endpoint using roles from jwt * test(delete): add test for non-admin user * perf(image): enable profile image to be nullable * feat(api): add support for refresh tokens Add RefreshToken endpoint and pass access/refresh token pairs via HttpOnly cookies * perf(update user): get user id exclusively from route * docs: add sequence flow diagrams for different scenarios * perf(refresh-token): reduce the no generated refresh tokens Generate refresh tokens only for specific scenarios, not for every kind of request: - valid refresh/access token pair provided - refresh/access token pair not already used to generate a new access token - refresh token not expired - access token expired - etc. * build(docker): add dockerfile for quetzalcoatl microservice * perf(refresh token): add trigger for deleting stale tokens * perf(refresh token): replace int with timespan for token lifetimes * perf(config): use IOptions * feat: add health check endpoint * build(docker): add docker-compose * perf(docker): assure that db is started before the api * feat: init enki as abp ddd project * feat(domain): define basic domain entities * feat(hermes): init delete/get download link * feat(hermes): finish delete test * feat(hermes): finish get download link for test * docs: update architecture diagram * build(hermes): test grpc dapr with hermes * build: configure dapr grpc intercommunication * build: configure dapr sidecars ports * feat+test(problem): add create method in domain service * feat(problem): implement Create endpoint * feat(problem): implement GetList endpoint * misc(quetzalcoatl): add proposer role to admin seeding too * feat(get-problems): implement retrieve problems related endpoints * feat(problem): implement Update endpoint * perf(pb-update): move domain logic to manager Delegate related domain logic to a dedicated method from ProblemManager * build: update dockerfiles for enki/quetzalcoatl * build: update dockerfiles for hermes/enki * build(asgard): add docker-compose support * feat(enki): implement grpc client for hermes conn * fix(docker): support mongodb transactions Add mongodb container as replica set to support transactions * feat(tests): implement add test support * style: remove unused imports * refactor(problem): remove programming languages Programming Languages are a concept more related to submissions than problem entities as they are not wielding any notable influence upon the latter * feat(test): implement update test * feat(test): implement delete test * perf(problem): remove number of tests dedicated field The number of tests can be easily yielded from the length of the tests collection * feat(test): add download urls to test entity Provide download urls retrieved from the TestService whenever a new test is created or an existing one is updated * feat(problem): implement support for publishing * perf(test): add download URLs in DTOs Include test download URLs in responses for proposer endpoints * feat(problem): implement eval metadata retrieval endpoint * fix: allow problem to be unpublished and use streams for upload - Allow a problem which is about to be unpublished to be edited (IsPublished field from DTO is false) - Replace IFormFile with IRemoteStreamContent (ABP native file handling support for multipart/form-data) * test(dapr): test enki-hermes integration with self-hosted dapr * refactor: add logging support to main services * build: test dapr mesh on docker compose
* feat(anubis): add init proj structure and diagram * feat(anubis): connect with pubsub dapr self-hosted - Connect with AWS SNS pubsub dapr component - Fix handler not found for topic * perf(docker): update dapr version Add redis pubsub dapr support (AWS SNS is not currently working) * feat: init pubsub flow between thor and anubis * docs(thor): add flow diagram for submissions * feat(thor): implement basic pending submission pub * perf(thor): add submission missing fields and dtos * feat(submission): implement basic eval flow - Implement submission flow between thor, anubis and enki - Add sub for thor to eval topic * feat(anubis): init languages config from toml file * build: add judge0 to docker compose * perf: init rocket and log4rs for new anubis architecture There is no more need of thor and dapr pubsub as anubis will directly communicate via dapr service invocation with judge0 * feat: add migrations for submissions * feat: add jwt middleware * feat(create-submission): add migrations and related infr insertion logic * feat(submit): add logic for retrieving eval metadata in anubis * feat(create-submission): get test contents and post batch submission * feat(create-submission): start implementing mapping to db entities obs: submissions are marked as pending in db after creation and only testcases tokens are added too (the actual polling will be implemented in a dedicated cron job) needs-fix: submission testcases not storing in db (serialization issue) * fix(enki): assign valid IDs for new tests and allow retrieval of eval metadata for unpublished problems * feat(create-submission): finish saving pending submission to db * perf(err-handling): add custom errors and catchers * refactor(anubis): add console logger appender * feat(eval-job): start implementing cron job for evaluation - implement the logic for retrieving the newly evaluated submission test cases from judge0 for the pending submissions - add expected_score field for submission test case (that retrieved from enki's eval metadata) * fix(anubis): add missing libs to dockerfile * build(anubis): add db url env var for anubis docker container instance * build: add dummy values for env templates * feat(eval-job): finish cron job for evaluation * feat(infr): add support for finding a submission by id * feat(get-submission): implement endpoint for retrieving submission with testcases by id * fix(get-submission): return not found for invalid submission id * feat(get-submissions): implement endpoint/repo method for retrieving all submissions * feat(get-submissions): add sorting and pagination * feat(get-submissions): add filtering * feat(dapr): add dapr state store as redis cache - Add caching support for get eval metadata and input/output tests download in anubis * feat(anubis): add business rules guards on endpoints - restrict submissions sending and viewing depending on the current logged in user - store problems eval metadata in db besides cache for improved filtering capabilities when retrieving submissions based on the current user * refactor(anubis): improve logging for all endpoints * fix(anubis-docker): deploy anubis to docker container and check integration with asgard - disable ssl certificates validation - listen to 0.0.0.0 from inside the container * build(docker): add every env file to dockerignore
* fix(enki): allow problem to be published iff the sum of its tests scores is 100 * refactor(enki): use more conventional REST paths for endpoints and remove unused projs * feat(odin): add http rerouting using envoy proxy * feat(odin): add ssl termination - add SSL termination for a newly exposed port and reroute traffic directly to services - keep the HTTP listener which redirects traffic to Dapr Sidecars - bug: quetzalcoatl app id not found for now using Dapr - bug: HTTPS certificate not verified despite being added to CA Certs * feat(odin): restrict access to eval-metadata - deny access to enki eval-metadata endpoint using lua filter * build: add github actions for publishing
* Hermes Tests Service v1.0 (#1) Existing functionalities: - gRPC methods for upload/download tests to Firebase Storage - simulate cache using a temporary disk storage - send/receive tests over/from network as stream of chunked zip archives - store decompressed versions of tests on Firebase Storage * Quetzalcoatl Custom JWT Auth (#2) * feat: add init project structure * feat(identity): scaffold db for default identity * docs(auth): add architecture diagram * feat(config): add jwt config * perf: move endpoints to separate proj than di * feat: add migrations for default identity * feat(infr): add new migrations for custom user * feat(domain): add user entity and jwt config * perf(di): add custom user to identity auth * feat(test): scaffold integration tests * feat(app): implement handlers required by auth -CreateUser -ValidateUserCredentials -GenerateJwtToken * refactor(test): split test proj between integration and unit * perf(mapper): use automapper for register endpoint * perf(app): organize handlers based on vertical slice * feat(test): add fixture for test containers * feat(test): create integration test for register endpoint * perf(api): refactor endpoints to improve readability * fix(test): remove sqlconnection error and speed up test containers - Declare additional IServiceCollection extension methods for ensuring db created and removing already existing DbContextOptions<AppDbContext> (not AppDbContext!!) - Use the latest MsSql Docker Image * feat(di): add default swagger support * test: remove unit tests proj As Fastendpoints Commands are hard to unit test outside of an Endpoint, it is enough to have only Integration tests to cover everything * test(login): add integration test for login * style: rename username prop and move jwt mapper here * docs(swagger): add swagger support * feat(users): scaffold CRUD users api * style(auth): move validators to separate files * feat(users): add get all endpoint * test(users): add integration tests for get all endpoint * feat(users): add get user endpoint * test(users): add integration test for get user endpoint * style(tests): add regions to enhance readability * style: replace local usings with global usings * perf(user): add guid as pk for application user * feat(update): create update user endpoint * test(update): create integration tests for update endpoint * misc: update .gitignore * perf(user manager): use predefined methods instead of LINQ for finding by id * feat(api): create delete endpoint * style: add regions and update global usings * test(delete): create integration tests for delete endpoint * docs(swagger): add summaries for Identity CRUD endpoints * style(binding): remove redundant bind from attribute for models Id * perf(update); enable request fields to be nullable In order to allow partial update for user profile (similar to patch), enable validation/mapping rules only when a certain request field is provided * style: add global usings * test(update): add integration test for partial update * style: format code * style: adjust namespace for jwt config * feat(user): add support for fullname and bio fields * test(user): update integration tests to support fullname/bio fields * feat(image): add support for user profile image * test(register): update integration tests to support profile image * perf(picture): enable support for picture in login endpoint Enable eager loading in db context for profile picture in order to be able to retrieve the image id for the download URL * feat(build): add migrations for profile picture * test(login): update login tests for picture Replace register endpoint invocation with the user manager directly in order to mitigate the tight coupling between tests and the QAPI * feat(picture): update the rest of endpoints to support profile picture * test(picture): update the rest of tests to support profile picture * feat(image): add get image endpoint * test(image): add integration test for get image endpoint * perf(image): create image repository * style: format code * perf: add argument null guards to ctors * feat(logger): configure serilog * misc: add Logs to .gitignore * perf: add logging to endpoints * fix(logger): replace bootstrap logger with default Bug related to integration tests fails * feat(update): add permission guard to endpoint Allow users to edit only their own profile, otherwise throw unauthorized * test: add integration test for self update permission * perf(update): replace unauthorized with forbidden * perf: use default identity role * perf: set model fields as init-only * feat: add seed data for roles and admin user * test(get all): remove users already added by seeding * perf(delete): restrict access to endpoint using roles from jwt * test(delete): add test for non-admin user * perf(image): enable profile image to be nullable * feat(api): add support for refresh tokens Add RefreshToken endpoint and pass access/refresh token pairs via HttpOnly cookies * perf(update user): get user id exclusively from route * docs: add sequence flow diagrams for different scenarios * Enki Problems Microservice + Dapr on Docker Compose integration (#4) * feat: add init project structure * feat(identity): scaffold db for default identity * docs(auth): add architecture diagram * feat(config): add jwt config * perf: move endpoints to separate proj than di * feat: add migrations for default identity * feat(infr): add new migrations for custom user * feat(domain): add user entity and jwt config * perf(di): add custom user to identity auth * feat(test): scaffold integration tests * feat(app): implement handlers required by auth -CreateUser -ValidateUserCredentials -GenerateJwtToken * refactor(test): split test proj between integration and unit * perf(mapper): use automapper for register endpoint * perf(app): organize handlers based on vertical slice * feat(test): add fixture for test containers * feat(test): create integration test for register endpoint * perf(api): refactor endpoints to improve readability * fix(test): remove sqlconnection error and speed up test containers - Declare additional IServiceCollection extension methods for ensuring db created and removing already existing DbContextOptions<AppDbContext> (not AppDbContext!!) - Use the latest MsSql Docker Image * feat(di): add default swagger support * test: remove unit tests proj As Fastendpoints Commands are hard to unit test outside of an Endpoint, it is enough to have only Integration tests to cover everything * test(login): add integration test for login * style: rename username prop and move jwt mapper here * docs(swagger): add swagger support * feat(users): scaffold CRUD users api * style(auth): move validators to separate files * feat(users): add get all endpoint * test(users): add integration tests for get all endpoint * feat(users): add get user endpoint * test(users): add integration test for get user endpoint * style(tests): add regions to enhance readability * style: replace local usings with global usings * perf(user): add guid as pk for application user * feat(update): create update user endpoint * test(update): create integration tests for update endpoint * misc: update .gitignore * perf(user manager): use predefined methods instead of LINQ for finding by id * feat(api): create delete endpoint * style: add regions and update global usings * test(delete): create integration tests for delete endpoint * docs(swagger): add summaries for Identity CRUD endpoints * style(binding): remove redundant bind from attribute for models Id * perf(update); enable request fields to be nullable In order to allow partial update for user profile (similar to patch), enable validation/mapping rules only when a certain request field is provided * style: add global usings * test(update): add integration test for partial update * style: format code * style: adjust namespace for jwt config * feat(user): add support for fullname and bio fields * test(user): update integration tests to support fullname/bio fields * feat(image): add support for user profile image * test(register): update integration tests to support profile image * perf(picture): enable support for picture in login endpoint Enable eager loading in db context for profile picture in order to be able to retrieve the image id for the download URL * feat(build): add migrations for profile picture * test(login): update login tests for picture Replace register endpoint invocation with the user manager directly in order to mitigate the tight coupling between tests and the QAPI * feat(picture): update the rest of endpoints to support profile picture * test(picture): update the rest of tests to support profile picture * feat(image): add get image endpoint * test(image): add integration test for get image endpoint * perf(image): create image repository * style: format code * perf: add argument null guards to ctors * feat(logger): configure serilog * misc: add Logs to .gitignore * perf: add logging to endpoints * fix(logger): replace bootstrap logger with default Bug related to integration tests fails * feat(update): add permission guard to endpoint Allow users to edit only their own profile, otherwise throw unauthorized * test: add integration test for self update permission * perf(update): replace unauthorized with forbidden * perf: use default identity role * perf: set model fields as init-only * feat: add seed data for roles and admin user * test(get all): remove users already added by seeding * perf(delete): restrict access to endpoint using roles from jwt * test(delete): add test for non-admin user * perf(image): enable profile image to be nullable * feat(api): add support for refresh tokens Add RefreshToken endpoint and pass access/refresh token pairs via HttpOnly cookies * perf(update user): get user id exclusively from route * docs: add sequence flow diagrams for different scenarios * perf(refresh-token): reduce the no generated refresh tokens Generate refresh tokens only for specific scenarios, not for every kind of request: - valid refresh/access token pair provided - refresh/access token pair not already used to generate a new access token - refresh token not expired - access token expired - etc. * build(docker): add dockerfile for quetzalcoatl microservice * perf(refresh token): add trigger for deleting stale tokens * perf(refresh token): replace int with timespan for token lifetimes * perf(config): use IOptions * feat: add health check endpoint * build(docker): add docker-compose * perf(docker): assure that db is started before the api * feat: init enki as abp ddd project * feat(domain): define basic domain entities * feat(hermes): init delete/get download link * feat(hermes): finish delete test * feat(hermes): finish get download link for test * docs: update architecture diagram * build(hermes): test grpc dapr with hermes * build: configure dapr grpc intercommunication * build: configure dapr sidecars ports * feat+test(problem): add create method in domain service * feat(problem): implement Create endpoint * feat(problem): implement GetList endpoint * misc(quetzalcoatl): add proposer role to admin seeding too * feat(get-problems): implement retrieve problems related endpoints * feat(problem): implement Update endpoint * perf(pb-update): move domain logic to manager Delegate related domain logic to a dedicated method from ProblemManager * build: update dockerfiles for enki/quetzalcoatl * build: update dockerfiles for hermes/enki * build(asgard): add docker-compose support * feat(enki): implement grpc client for hermes conn * fix(docker): support mongodb transactions Add mongodb container as replica set to support transactions * feat(tests): implement add test support * style: remove unused imports * refactor(problem): remove programming languages Programming Languages are a concept more related to submissions than problem entities as they are not wielding any notable influence upon the latter * feat(test): implement update test * feat(test): implement delete test * perf(problem): remove number of tests dedicated field The number of tests can be easily yielded from the length of the tests collection * feat(test): add download urls to test entity Provide download urls retrieved from the TestService whenever a new test is created or an existing one is updated * feat(problem): implement support for publishing * perf(test): add download URLs in DTOs Include test download URLs in responses for proposer endpoints * feat(problem): implement eval metadata retrieval endpoint * fix: allow problem to be unpublished and use streams for upload - Allow a problem which is about to be unpublished to be edited (IsPublished field from DTO is false) - Replace IFormFile with IRemoteStreamContent (ABP native file handling support for multipart/form-data) * test(dapr): test enki-hermes integration with self-hosted dapr * refactor: add logging support to main services * build: test dapr mesh on docker compose * Anubis (#5) * feat(anubis): add init proj structure and diagram * feat(anubis): connect with pubsub dapr self-hosted - Connect with AWS SNS pubsub dapr component - Fix handler not found for topic * perf(docker): update dapr version Add redis pubsub dapr support (AWS SNS is not currently working) * feat: init pubsub flow between thor and anubis * docs(thor): add flow diagram for submissions * feat(thor): implement basic pending submission pub * perf(thor): add submission missing fields and dtos * feat(submission): implement basic eval flow - Implement submission flow between thor, anubis and enki - Add sub for thor to eval topic * feat(anubis): init languages config from toml file * build: add judge0 to docker compose * perf: init rocket and log4rs for new anubis architecture There is no more need of thor and dapr pubsub as anubis will directly communicate via dapr service invocation with judge0 * feat: add migrations for submissions * feat: add jwt middleware * feat(create-submission): add migrations and related infr insertion logic * feat(submit): add logic for retrieving eval metadata in anubis * feat(create-submission): get test contents and post batch submission * feat(create-submission): start implementing mapping to db entities obs: submissions are marked as pending in db after creation and only testcases tokens are added too (the actual polling will be implemented in a dedicated cron job) needs-fix: submission testcases not storing in db (serialization issue) * fix(enki): assign valid IDs for new tests and allow retrieval of eval metadata for unpublished problems * feat(create-submission): finish saving pending submission to db * perf(err-handling): add custom errors and catchers * refactor(anubis): add console logger appender * feat(eval-job): start implementing cron job for evaluation - implement the logic for retrieving the newly evaluated submission test cases from judge0 for the pending submissions - add expected_score field for submission test case (that retrieved from enki's eval metadata) * fix(anubis): add missing libs to dockerfile * build(anubis): add db url env var for anubis docker container instance * build: add dummy values for env templates * feat(eval-job): finish cron job for evaluation * feat(infr): add support for finding a submission by id * feat(get-submission): implement endpoint for retrieving submission with testcases by id * fix(get-submission): return not found for invalid submission id * feat(get-submissions): implement endpoint/repo method for retrieving all submissions * feat(get-submissions): add sorting and pagination * feat(get-submissions): add filtering * feat(dapr): add dapr state store as redis cache - Add caching support for get eval metadata and input/output tests download in anubis * feat(anubis): add business rules guards on endpoints - restrict submissions sending and viewing depending on the current logged in user - store problems eval metadata in db besides cache for improved filtering capabilities when retrieving submissions based on the current user * refactor(anubis): improve logging for all endpoints * fix(anubis-docker): deploy anubis to docker container and check integration with asgard - disable ssl certificates validation - listen to 0.0.0.0 from inside the container * build(docker): add every env file to dockerignore * Odin (#6) * fix(enki): allow problem to be published iff the sum of its tests scores is 100 * refactor(enki): use more conventional REST paths for endpoints and remove unused projs * feat(odin): add http rerouting using envoy proxy * feat(odin): add ssl termination - add SSL termination for a newly exposed port and reroute traffic directly to services - keep the HTTP listener which redirects traffic to Dapr Sidecars - bug: quetzalcoatl app id not found for now using Dapr - bug: HTTPS certificate not verified despite being added to CA Certs * feat(odin): restrict access to eval-metadata - deny access to enki eval-metadata endpoint using lua filter * build: add github actions for publishing * build: update workflows (secrets and paths) * build: mv workflows to parent folder * build: specify repo owner in image id * fix(workflows): fix pull_request identation * build: try to fix invalid docker tag id * build: try to fix invalid docker tag id * fix(workflows): specify the correct path triggers * fix(worflows): specify correct context for build * style: add service name in each workflow name * build: add prod deployment workflow * fix(workflows): fix docker compose name mispell * ops: add global asgard profile for docker compose * fix(workflows): add ghcr login to prod workflow * fix(prod-workflow): add env var for registry * fix(prod-workflow): add ghcr login inside droplet * fix(prod): login into ghcr using password-stdin * build: update images name for dapr and odin * build: add prod dedicated docker compose file * fix(prod): escape quotes in enki mongodb cmd * build(prod-worflow): disable for push on develop * build(prod-workflow): disable for push on master
bug: Kotlin cannot be currently used due to compilation time limit exceeded
- split submission batch into chunks before sending it to Judge0 to avoid CONNECTION_RESET_ERROR - remove input/output tests contents from logs - add more suggestive HTTP error messages
* refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config
* refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * refactor(release-please): change tag separator to slash * refactor(release-please): update target branch
* perf(quetzalcoatl): extract cors origins in envvar * refactor(anubis): fix api endpoint mispell * feat(cors): enable cors policy for anubis, enki and quetzalcoatl * perf(quetzalcoatl): simplify refresh token logic - now only a valid refresh token is required in order to issue new access tokens, as keeping the stale access tokens in sync with the refresh token using cookies is too intricate * perf(odin): add lua filter for appending access/refresh tokens from/to requests/responses * perf(odin): increase expiry time for newly created access tokens - the expiry time is currently hardcoded in the envoy.yaml file as I haven't figured it out how to use env vars here yet * perf(register): set auth tokens as samesite lax * perf(docker): do not restart containers if stopped * docs: add fastendpoints logo to diagram * perf(quetzalcoatl): update user dtos to use ProfilePictureId instead of ProfilePictureUrl * refactor(quetzalcoatl): set cookies as secure on prod * perf(prod): update SameSite attribute for cookies to None * fix(quetzalcoatl): check profile picture not to be null before adding its id to dtos * feat(users): add support for filtering-sorting-pagination for users endpoint * perf(users): define all fsp params as optional * feat: Add logging and error handling to refresh token endpoint Added logging statements to display the request path and HTTP method. Also added error handling for cases where cookies are missing or invalid, with appropriate response headers. The response headers now include `access-control-allow-credentials` and `access-control-allow-origin` to enable CORS support. * feat(images): allow anonymous users to access the image endpoint * perf(odin): add https support * feat(compose): add service for midgard-web * ops: add pantheonix profile to services * feat(roles): add endpoints for roles management * fix(odin): remove certs dependency as they are already set in lb * fix(odin): remove https redirection * fix(remove role): add role information to user response * refactor(validators): update validation rules and add constants for user registration and update validators * refactor(users): update GetAllUsersEndpoint to filter users by username instead of name * perf(enki): add authorization to GetListAsync and GetAsync methods * fix(quetzalcoatl): get total count of items after filtering for correct user pagination on GetAll endpoint * fix(enki): count items after filtering for correct pagination of problems (both published and unpublished) * feat(enki): add delete problem endpoint (propagate deletion event against hermes too) * style(enki): format code * fix(enki): allow proposer to keep the same name for an existing problem on problem update endpoint * feat(anubis): add endpoint for retrieving the highest score submissions per user (and problem if specified) * feat(enki+anubis): add pubsub support for eval metadata retrieval to improve performance Add RabbitMQ pub-sub using Dapr for publishing eval metadata related events when problems/tests are upserted/deleted in order to mitigate the overhead from Anubis submission evaluation * feat(docker-compose): add volumes for dapr redis and rabbitmq logs * fix(anubis): update tests PK as the composition between id and problem_id * feat(anubis): add create/get test case dapr client methods * perf(anubis): improve http errors format using json * perf(anubis): add problem name to get all submissions endpoint response * feat(anubis-judge0): add nginx lb between anubis and judge0 replicas * fix(anubis): configure CORS policy for Rocket * perf(anubis): add problem name to get submission by id endpoint response * ops(asgard): configure memory limits/reservations for docker containers * perf(anubis): add is_published field to submissions dtos * ops(compose): update resource limits and reservations * fix(submission source code): show submission source code iff problem has been solved previously by user * fix(anubis): remove unused import * ops: update docker-compose.prod.yaml * ops(anubis): scale up evaluator with more judge0 replicas * ops(anubis): add rabbitmq instance to prod * fix(anubis): add cors preflight catcher * fix(ci/cd): add support to auto-tagging both for git and docker * fix(ci/cd): update digitalocean production environment only when pushing to master * fix(ci/cd): disable autogenerated tags for now * fix(ci/cd): remove github action for autogenerated tags * fix(ci/cd): disable triggers on pull-request * ops(anubis): use cargo-chef for improving docker image building via deps caching (#18) Co-authored-by: WarriorsSami <[email protected]> * Polish CI pipelines for building/testing and testing environments (#19) * ops(ci): add ci pipelines for building and testing quetzalcoatl and enki * fix(ci): remove caching layer from quetzalcoatl and enki pipelines * fix(ci): cd into quetzalcoatl and enki microservices projects before executing pipelines * fix(ci): pwd for quetzalcoatl in pipeline * fix(ci-quetzalcoatl): specify path to project when building and testing * fix(ci-quetzalcoatl): restore tests project too * misc(quetzalcoatl): update .net sdk to 8 (and packages too) * fix(quetzalcoatl): configure testing environment * fix+test(quetzalcoatl): remove obsolete tests and add new type formatters for response parsing * fix(quetzalcoatl): configure testing environment for ci pipeline * fix(quetzalcoatl): use .net 8 sdk in ci pipeline * fix(quetzalcoatl): downgrade back to .net 7 as authorization failed to work in .net 8 * test(quetzalcoatl): remove seed data for get all users test * style: format code * fix(quetzalcoatl): enable MARS for mssql dsn in integration tests * style: remove debuggin prints * fix+ops(enki): prepare test environment for ci pipeline * ops(hermes): setup testing environment for ci pipeline * fix(hermes): remove format and analyze steps from ci pipeline * fix(hermes): cd into microservice project before running tests * fix(hermes): create log file for tests * ops: add pull-request triggers for ci pipelines and prepend ci logic to already existing cd pipelines * ops(anubis): setup ci pipeline for testing * refactor+ops: specify project dir as default per pipeline * fix(anubis): place cache action after rust setup in ci pipeline * fix(quetzalcoatl): specify path to test project .csproj in the testing action of the ci pipeline * test+ops(anubis): setup local environment for integration testing using docker compose * fix(anubis): use correct path to testing environment compose in ci pipeline * test(anubis): setup seeding data for integration tests * test(anubis): setup cache stub as go CRUD API wrapper over Redis for integration testing * test(anubis): add tests for get submission(s) endpoints and setup nextest * ops(anubis): specify nextest version to be compatible with rustc version in ci pipeline * ops(anubis): specify nextest version to be compatible with rustc version in ci pipeline * test(anubis): add tests for get the highest score submissions and create submission endpoints * ops: disable digitalocean deployment pipeline ops: disable digitalocean deployment pipeline ops: disable digitalocean deployment pipeline --------- Co-authored-by: WarriorsSami <[email protected]> * fix(anubis cd): specify version for cargo nextest * fix(cd pipelines): make deploy job depend upon the build one * fix(anubis cd): add docker compose step for setting up testing environment * Seeding script for Asgard local environment (#20) * refactor(docker-compose): remove deprecated version key and expose new port for debugging dapr redis * feat(seeder): add support for deserializing fixtures yaml config file into golang struct * feat(seeder): implement login method in http client and return jwt access token iff logged in successfully * feat(seeder): implement create problem method * feat(seeder): implement create tests method * feat(seeder): implement publish problem method * feat(seeder): implement submission sending perf: use goroutines to leverage parallel processing of http requests perf: use errgroup to report errors encountered inside goroutines * perf(seeder): execute seeding worker dynamically using docker compose * fix(seeder): add compose.override file for seeder * Add PR Verify Pipelines (#22) * refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * Devops/ci cd enhancement (#23) * refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * refactor(release-please): change tag separator to slash * refactor(release-please): update target branch --------- Co-authored-by: WarriorsSami <[email protected]>
* refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * refactor(release-please): change tag separator to slash * refactor(release-please): update target branch * fix(release-please): correct config according to release please schema
# Conflicts: # release-please-config.json
DevOps improvements (#24) * perf(quetzalcoatl): extract cors origins in envvar * refactor(anubis): fix api endpoint mispell * feat(cors): enable cors policy for anubis, enki and quetzalcoatl * perf(quetzalcoatl): simplify refresh token logic - now only a valid refresh token is required in order to issue new access tokens, as keeping the stale access tokens in sync with the refresh token using cookies is too intricate * perf(odin): add lua filter for appending access/refresh tokens from/to requests/responses * perf(odin): increase expiry time for newly created access tokens - the expiry time is currently hardcoded in the envoy.yaml file as I haven't figured it out how to use env vars here yet * perf(register): set auth tokens as samesite lax * perf(docker): do not restart containers if stopped * docs: add fastendpoints logo to diagram * perf(quetzalcoatl): update user dtos to use ProfilePictureId instead of ProfilePictureUrl * refactor(quetzalcoatl): set cookies as secure on prod * perf(prod): update SameSite attribute for cookies to None * fix(quetzalcoatl): check profile picture not to be null before adding its id to dtos * feat(users): add support for filtering-sorting-pagination for users endpoint * perf(users): define all fsp params as optional * feat: Add logging and error handling to refresh token endpoint Added logging statements to display the request path and HTTP method. Also added error handling for cases where cookies are missing or invalid, with appropriate response headers. The response headers now include `access-control-allow-credentials` and `access-control-allow-origin` to enable CORS support. * feat(images): allow anonymous users to access the image endpoint * perf(odin): add https support * feat(compose): add service for midgard-web * ops: add pantheonix profile to services * feat(roles): add endpoints for roles management * fix(odin): remove certs dependency as they are already set in lb * fix(odin): remove https redirection * fix(remove role): add role information to user response * refactor(validators): update validation rules and add constants for user registration and update validators * refactor(users): update GetAllUsersEndpoint to filter users by username instead of name * perf(enki): add authorization to GetListAsync and GetAsync methods * fix(quetzalcoatl): get total count of items after filtering for correct user pagination on GetAll endpoint * fix(enki): count items after filtering for correct pagination of problems (both published and unpublished) * feat(enki): add delete problem endpoint (propagate deletion event against hermes too) * style(enki): format code * fix(enki): allow proposer to keep the same name for an existing problem on problem update endpoint * feat(anubis): add endpoint for retrieving the highest score submissions per user (and problem if specified) * feat(enki+anubis): add pubsub support for eval metadata retrieval to improve performance Add RabbitMQ pub-sub using Dapr for publishing eval metadata related events when problems/tests are upserted/deleted in order to mitigate the overhead from Anubis submission evaluation * feat(docker-compose): add volumes for dapr redis and rabbitmq logs * fix(anubis): update tests PK as the composition between id and problem_id * feat(anubis): add create/get test case dapr client methods * perf(anubis): improve http errors format using json * perf(anubis): add problem name to get all submissions endpoint response * feat(anubis-judge0): add nginx lb between anubis and judge0 replicas * fix(anubis): configure CORS policy for Rocket * perf(anubis): add problem name to get submission by id endpoint response * ops(asgard): configure memory limits/reservations for docker containers * perf(anubis): add is_published field to submissions dtos * ops(compose): update resource limits and reservations * fix(submission source code): show submission source code iff problem has been solved previously by user * fix(anubis): remove unused import * ops: update docker-compose.prod.yaml * ops(anubis): scale up evaluator with more judge0 replicas * ops(anubis): add rabbitmq instance to prod * fix(anubis): add cors preflight catcher * fix(ci/cd): add support to auto-tagging both for git and docker * fix(ci/cd): update digitalocean production environment only when pushing to master * fix(ci/cd): disable autogenerated tags for now * fix(ci/cd): remove github action for autogenerated tags * fix(ci/cd): disable triggers on pull-request * ops(anubis): use cargo-chef for improving docker image building via deps caching (#18) * Polish CI pipelines for building/testing and testing environments (#19) * ops(ci): add ci pipelines for building and testing quetzalcoatl and enki * fix(ci): remove caching layer from quetzalcoatl and enki pipelines * fix(ci): cd into quetzalcoatl and enki microservices projects before executing pipelines * fix(ci): pwd for quetzalcoatl in pipeline * fix(ci-quetzalcoatl): specify path to project when building and testing * fix(ci-quetzalcoatl): restore tests project too * misc(quetzalcoatl): update .net sdk to 8 (and packages too) * fix(quetzalcoatl): configure testing environment * fix+test(quetzalcoatl): remove obsolete tests and add new type formatters for response parsing * fix(quetzalcoatl): configure testing environment for ci pipeline * fix(quetzalcoatl): use .net 8 sdk in ci pipeline * fix(quetzalcoatl): downgrade back to .net 7 as authorization failed to work in .net 8 * test(quetzalcoatl): remove seed data for get all users test * style: format code * fix(quetzalcoatl): enable MARS for mssql dsn in integration tests * style: remove debuggin prints * fix+ops(enki): prepare test environment for ci pipeline * ops(hermes): setup testing environment for ci pipeline * fix(hermes): remove format and analyze steps from ci pipeline * fix(hermes): cd into microservice project before running tests * fix(hermes): create log file for tests * ops: add pull-request triggers for ci pipelines and prepend ci logic to already existing cd pipelines * ops(anubis): setup ci pipeline for testing * refactor+ops: specify project dir as default per pipeline * fix(anubis): place cache action after rust setup in ci pipeline * fix(quetzalcoatl): specify path to test project .csproj in the testing action of the ci pipeline * test+ops(anubis): setup local environment for integration testing using docker compose * fix(anubis): use correct path to testing environment compose in ci pipeline * test(anubis): setup seeding data for integration tests * test(anubis): setup cache stub as go CRUD API wrapper over Redis for integration testing * test(anubis): add tests for get submission(s) endpoints and setup nextest * ops(anubis): specify nextest version to be compatible with rustc version in ci pipeline * ops(anubis): specify nextest version to be compatible with rustc version in ci pipeline * test(anubis): add tests for get the highest score submissions and create submission endpoints * ops: disable digitalocean deployment pipeline ops: disable digitalocean deployment pipeline ops: disable digitalocean deployment pipeline --------- * fix(anubis cd): specify version for cargo nextest * fix(cd pipelines): make deploy job depend upon the build one * fix(anubis cd): add docker compose step for setting up testing environment * Seeding script for Asgard local environment (#20) * refactor(docker-compose): remove deprecated version key and expose new port for debugging dapr redis * feat(seeder): add support for deserializing fixtures yaml config file into golang struct * feat(seeder): implement login method in http client and return jwt access token iff logged in successfully * feat(seeder): implement create problem method * feat(seeder): implement create tests method * feat(seeder): implement publish problem method * feat(seeder): implement submission sending perf: use goroutines to leverage parallel processing of http requests perf: use errgroup to report errors encountered inside goroutines * perf(seeder): execute seeding worker dynamically using docker compose * fix(seeder): add compose.override file for seeder * Add PR Verify Pipelines (#22) * refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * Devops/ci cd enhancement (#23) * refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * refactor(release-please): change tag separator to slash * refactor(release-please): update target branch --------- Co-authored-by: WarriorsSami <[email protected]>
* DevOps improvements (#24) * perf(quetzalcoatl): extract cors origins in envvar * refactor(anubis): fix api endpoint mispell * feat(cors): enable cors policy for anubis, enki and quetzalcoatl * perf(quetzalcoatl): simplify refresh token logic - now only a valid refresh token is required in order to issue new access tokens, as keeping the stale access tokens in sync with the refresh token using cookies is too intricate * perf(odin): add lua filter for appending access/refresh tokens from/to requests/responses * perf(odin): increase expiry time for newly created access tokens - the expiry time is currently hardcoded in the envoy.yaml file as I haven't figured it out how to use env vars here yet * perf(register): set auth tokens as samesite lax * perf(docker): do not restart containers if stopped * docs: add fastendpoints logo to diagram * perf(quetzalcoatl): update user dtos to use ProfilePictureId instead of ProfilePictureUrl * refactor(quetzalcoatl): set cookies as secure on prod * perf(prod): update SameSite attribute for cookies to None * fix(quetzalcoatl): check profile picture not to be null before adding its id to dtos * feat(users): add support for filtering-sorting-pagination for users endpoint * perf(users): define all fsp params as optional * feat: Add logging and error handling to refresh token endpoint Added logging statements to display the request path and HTTP method. Also added error handling for cases where cookies are missing or invalid, with appropriate response headers. The response headers now include `access-control-allow-credentials` and `access-control-allow-origin` to enable CORS support. * feat(images): allow anonymous users to access the image endpoint * perf(odin): add https support * feat(compose): add service for midgard-web * ops: add pantheonix profile to services * feat(roles): add endpoints for roles management * fix(odin): remove certs dependency as they are already set in lb * fix(odin): remove https redirection * fix(remove role): add role information to user response * refactor(validators): update validation rules and add constants for user registration and update validators * refactor(users): update GetAllUsersEndpoint to filter users by username instead of name * perf(enki): add authorization to GetListAsync and GetAsync methods * fix(quetzalcoatl): get total count of items after filtering for correct user pagination on GetAll endpoint * fix(enki): count items after filtering for correct pagination of problems (both published and unpublished) * feat(enki): add delete problem endpoint (propagate deletion event against hermes too) * style(enki): format code * fix(enki): allow proposer to keep the same name for an existing problem on problem update endpoint * feat(anubis): add endpoint for retrieving the highest score submissions per user (and problem if specified) * feat(enki+anubis): add pubsub support for eval metadata retrieval to improve performance Add RabbitMQ pub-sub using Dapr for publishing eval metadata related events when problems/tests are upserted/deleted in order to mitigate the overhead from Anubis submission evaluation * feat(docker-compose): add volumes for dapr redis and rabbitmq logs * fix(anubis): update tests PK as the composition between id and problem_id * feat(anubis): add create/get test case dapr client methods * perf(anubis): improve http errors format using json * perf(anubis): add problem name to get all submissions endpoint response * feat(anubis-judge0): add nginx lb between anubis and judge0 replicas * fix(anubis): configure CORS policy for Rocket * perf(anubis): add problem name to get submission by id endpoint response * ops(asgard): configure memory limits/reservations for docker containers * perf(anubis): add is_published field to submissions dtos * ops(compose): update resource limits and reservations * fix(submission source code): show submission source code iff problem has been solved previously by user * fix(anubis): remove unused import * ops: update docker-compose.prod.yaml * ops(anubis): scale up evaluator with more judge0 replicas * ops(anubis): add rabbitmq instance to prod * fix(anubis): add cors preflight catcher * fix(ci/cd): add support to auto-tagging both for git and docker * fix(ci/cd): update digitalocean production environment only when pushing to master * fix(ci/cd): disable autogenerated tags for now * fix(ci/cd): remove github action for autogenerated tags * fix(ci/cd): disable triggers on pull-request * ops(anubis): use cargo-chef for improving docker image building via deps caching (#18) Co-authored-by: WarriorsSami <[email protected]> * Polish CI pipelines for building/testing and testing environments (#19) * ops(ci): add ci pipelines for building and testing quetzalcoatl and enki * fix(ci): remove caching layer from quetzalcoatl and enki pipelines * fix(ci): cd into quetzalcoatl and enki microservices projects before executing pipelines * fix(ci): pwd for quetzalcoatl in pipeline * fix(ci-quetzalcoatl): specify path to project when building and testing * fix(ci-quetzalcoatl): restore tests project too * misc(quetzalcoatl): update .net sdk to 8 (and packages too) * fix(quetzalcoatl): configure testing environment * fix+test(quetzalcoatl): remove obsolete tests and add new type formatters for response parsing * fix(quetzalcoatl): configure testing environment for ci pipeline * fix(quetzalcoatl): use .net 8 sdk in ci pipeline * fix(quetzalcoatl): downgrade back to .net 7 as authorization failed to work in .net 8 * test(quetzalcoatl): remove seed data for get all users test * style: format code * fix(quetzalcoatl): enable MARS for mssql dsn in integration tests * style: remove debuggin prints * fix+ops(enki): prepare test environment for ci pipeline * ops(hermes): setup testing environment for ci pipeline * fix(hermes): remove format and analyze steps from ci pipeline * fix(hermes): cd into microservice project before running tests * fix(hermes): create log file for tests * ops: add pull-request triggers for ci pipelines and prepend ci logic to already existing cd pipelines * ops(anubis): setup ci pipeline for testing * refactor+ops: specify project dir as default per pipeline * fix(anubis): place cache action after rust setup in ci pipeline * fix(quetzalcoatl): specify path to test project .csproj in the testing action of the ci pipeline * test+ops(anubis): setup local environment for integration testing using docker compose * fix(anubis): use correct path to testing environment compose in ci pipeline * test(anubis): setup seeding data for integration tests * test(anubis): setup cache stub as go CRUD API wrapper over Redis for integration testing * test(anubis): add tests for get submission(s) endpoints and setup nextest * ops(anubis): specify nextest version to be compatible with rustc version in ci pipeline * ops(anubis): specify nextest version to be compatible with rustc version in ci pipeline * test(anubis): add tests for get the highest score submissions and create submission endpoints * ops: disable digitalocean deployment pipeline ops: disable digitalocean deployment pipeline ops: disable digitalocean deployment pipeline --------- Co-authored-by: WarriorsSami <[email protected]> * fix(anubis cd): specify version for cargo nextest * fix(cd pipelines): make deploy job depend upon the build one * fix(anubis cd): add docker compose step for setting up testing environment * Seeding script for Asgard local environment (#20) * refactor(docker-compose): remove deprecated version key and expose new port for debugging dapr redis * feat(seeder): add support for deserializing fixtures yaml config file into golang struct * feat(seeder): implement login method in http client and return jwt access token iff logged in successfully * feat(seeder): implement create problem method * feat(seeder): implement create tests method * feat(seeder): implement publish problem method * feat(seeder): implement submission sending perf: use goroutines to leverage parallel processing of http requests perf: use errgroup to report errors encountered inside goroutines * perf(seeder): execute seeding worker dynamically using docker compose * fix(seeder): add compose.override file for seeder * Add PR Verify Pipelines (#22) * refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * Devops/ci cd enhancement (#23) * refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * refactor(release-please): change tag separator to slash * refactor(release-please): update target branch --------- Co-authored-by: WarriorsSami <[email protected]> * fix(release-please): use custom github pat --------- Co-authored-by: WarriorsSami <[email protected]>
* fix(release-please): use custom github pat * fix(ci-cd): remove redundant env vars from ci pipelines
…ated microservices branches
…nst dedicated microservices branches (#38) * refactor(ci-cd): define reusable workflows for pushing images to ghcr and docker hub * refactor(ci-cd): reconfigure ci pipelines as pr verify ones * refactor(ci-cd): integrate reusable workflows for image publishing in ci pipelines * fix(quetzalcoatl): pin explicit mssql tag for testcontainers in integration tests * fix: disable testing step in quetzalcoatl pipelines due to misbehaving mssql containers * feat(ci-cd): add release please pipeline and config * refactor(release-please): change tag separator to slash * refactor(release-please): update target branch * fix(release-please): correct config according to release please schema * perf(ci-cd): update pr verify pipelines to be triggered against dedicated microservices branches * fix(ci-cd): add tag to image metadata for docker hub step
# Conflicts: # .gitignore # quetzalcoatl-auth/.dockerignore # quetzalcoatl-auth/.gitignore # quetzalcoatl-auth/Api/Api.csproj # quetzalcoatl-auth/Api/Features/Auth/Login/Endpoint.cs # quetzalcoatl-auth/Api/Features/Auth/Login/Models.cs # quetzalcoatl-auth/Api/Features/Auth/Login/Summary.cs # quetzalcoatl-auth/Api/Features/Auth/RefreshToken/Endpoint.cs # quetzalcoatl-auth/Api/Features/Auth/RefreshToken/Mappers.cs # quetzalcoatl-auth/Api/Features/Auth/RefreshToken/Models.cs # quetzalcoatl-auth/Api/Features/Auth/Register/Endpoint.cs # quetzalcoatl-auth/Api/Features/Auth/Register/Summary.cs # quetzalcoatl-auth/Api/Features/Auth/Register/Validators.cs # quetzalcoatl-auth/Api/Features/Core/ApplicationUserExtensions.cs # quetzalcoatl-auth/Api/Features/Images/ImagesGroup.cs # quetzalcoatl-auth/Api/Features/Users/Core/UserDto.cs # quetzalcoatl-auth/Api/Features/Users/Delete/Endpoint.cs # quetzalcoatl-auth/Api/Features/Users/Get/Endpoint.cs # quetzalcoatl-auth/Api/Features/Users/Get/Mappers.cs # quetzalcoatl-auth/Api/Features/Users/Get/Models.cs # quetzalcoatl-auth/Api/Features/Users/GetAll/Endpoint.cs # quetzalcoatl-auth/Api/Features/Users/GetAll/Mappers.cs # quetzalcoatl-auth/Api/Features/Users/GetAll/Models.cs # quetzalcoatl-auth/Api/Features/Users/GetAll/Summary.cs # quetzalcoatl-auth/Api/Features/Users/Update/Endpoint.cs # quetzalcoatl-auth/Api/Features/Users/Update/Mappers.cs # quetzalcoatl-auth/Api/Features/Users/Update/Models.cs # quetzalcoatl-auth/Api/Features/Users/Update/Summary.cs # quetzalcoatl-auth/Api/Features/Users/Update/Validators.cs # quetzalcoatl-auth/Api/Usings.cs # quetzalcoatl-auth/Application/Features/Users/CreateUser/Handler.cs # quetzalcoatl-auth/Application/Usings.cs # quetzalcoatl-auth/Bootstrapper/Bootstrapper.csproj # quetzalcoatl-auth/Bootstrapper/Extensions/ServiceCollectionExtensions.cs # quetzalcoatl-auth/Bootstrapper/Extensions/WebApplicationExtensions.cs # quetzalcoatl-auth/Bootstrapper/Program.cs # quetzalcoatl-auth/Bootstrapper/Usings.cs # quetzalcoatl-auth/Bootstrapper/appsettings.json # quetzalcoatl-auth/Dockerfile # quetzalcoatl-auth/Domain/Entities/RefreshToken.cs # quetzalcoatl-auth/Domain/Usings.cs # quetzalcoatl-auth/Infrastructure/ApplicationDbContext.cs # quetzalcoatl-auth/Infrastructure/Infrastructure.csproj # quetzalcoatl-auth/Infrastructure/Migrations/20230315154826_AddGuidAsPKForIdentityUser.cs # quetzalcoatl-auth/Infrastructure/Migrations/20230319104138_AddProfileImageToIdentityUser.cs # quetzalcoatl-auth/Infrastructure/Migrations/20230525105824_UpdateRefreshTokenEntity.cs # quetzalcoatl-auth/Infrastructure/Migrations/ApplicationDbContextModelSnapshot.cs # quetzalcoatl-auth/Infrastructure/Triggers/DeleteStaleRefreshTokens.cs # quetzalcoatl-auth/Infrastructure/Usings.cs # quetzalcoatl-auth/Tests.Integration/Api/Features/Auth/LoginEndpointTests.cs # quetzalcoatl-auth/Tests.Integration/Api/Features/Auth/RegisterEndpointTests.cs # quetzalcoatl-auth/Tests.Integration/Api/Features/Images/GetImageEndpointTests.cs # quetzalcoatl-auth/Tests.Integration/Api/Features/Users/DeleteEndpointTests.cs # quetzalcoatl-auth/Tests.Integration/Api/Features/Users/GetAllEndpointTests.cs # quetzalcoatl-auth/Tests.Integration/Api/Features/Users/GetEndpointTests.cs # quetzalcoatl-auth/Tests.Integration/Api/Features/Users/UpdateEndpointTests.cs # quetzalcoatl-auth/Tests.Integration/Core/ApiWebFactory.cs # quetzalcoatl-auth/Tests.Integration/Usings.cs
WarriorsSami
added a commit
that referenced
this pull request
Sep 24, 2024
* feat: add init project structure * feat(identity): scaffold db for default identity * docs(auth): add architecture diagram * feat(config): add jwt config * perf: move endpoints to separate proj than di * feat: add migrations for default identity * feat(infr): add new migrations for custom user * feat(domain): add user entity and jwt config * perf(di): add custom user to identity auth * feat(test): scaffold integration tests * feat(app): implement handlers required by auth -CreateUser -ValidateUserCredentials -GenerateJwtToken * refactor(test): split test proj between integration and unit * perf(mapper): use automapper for register endpoint * perf(app): organize handlers based on vertical slice * feat(test): add fixture for test containers * feat(test): create integration test for register endpoint * perf(api): refactor endpoints to improve readability * fix(test): remove sqlconnection error and speed up test containers - Declare additional IServiceCollection extension methods for ensuring db created and removing already existing DbContextOptions<AppDbContext> (not AppDbContext!!) - Use the latest MsSql Docker Image * feat(di): add default swagger support * test: remove unit tests proj As Fastendpoints Commands are hard to unit test outside of an Endpoint, it is enough to have only Integration tests to cover everything * test(login): add integration test for login * style: rename username prop and move jwt mapper here * docs(swagger): add swagger support * feat(users): scaffold CRUD users api * style(auth): move validators to separate files * feat(users): add get all endpoint * test(users): add integration tests for get all endpoint * feat(users): add get user endpoint * test(users): add integration test for get user endpoint * style(tests): add regions to enhance readability * style: replace local usings with global usings * perf(user): add guid as pk for application user * feat(update): create update user endpoint * test(update): create integration tests for update endpoint * misc: update .gitignore * perf(user manager): use predefined methods instead of LINQ for finding by id * feat(api): create delete endpoint * style: add regions and update global usings * test(delete): create integration tests for delete endpoint * docs(swagger): add summaries for Identity CRUD endpoints * style(binding): remove redundant bind from attribute for models Id * perf(update); enable request fields to be nullable In order to allow partial update for user profile (similar to patch), enable validation/mapping rules only when a certain request field is provided * style: add global usings * test(update): add integration test for partial update * style: format code * style: adjust namespace for jwt config * feat(user): add support for fullname and bio fields * test(user): update integration tests to support fullname/bio fields * feat(image): add support for user profile image * test(register): update integration tests to support profile image * perf(picture): enable support for picture in login endpoint Enable eager loading in db context for profile picture in order to be able to retrieve the image id for the download URL * feat(build): add migrations for profile picture * test(login): update login tests for picture Replace register endpoint invocation with the user manager directly in order to mitigate the tight coupling between tests and the QAPI * feat(picture): update the rest of endpoints to support profile picture * test(picture): update the rest of tests to support profile picture * feat(image): add get image endpoint * test(image): add integration test for get image endpoint * perf(image): create image repository * style: format code * perf: add argument null guards to ctors * feat(logger): configure serilog * misc: add Logs to .gitignore * perf: add logging to endpoints * fix(logger): replace bootstrap logger with default Bug related to integration tests fails * feat(update): add permission guard to endpoint Allow users to edit only their own profile, otherwise throw unauthorized * test: add integration test for self update permission * perf(update): replace unauthorized with forbidden * perf: use default identity role * perf: set model fields as init-only * feat: add seed data for roles and admin user * test(get all): remove users already added by seeding * perf(delete): restrict access to endpoint using roles from jwt * test(delete): add test for non-admin user * perf(image): enable profile image to be nullable * feat(api): add support for refresh tokens Add RefreshToken endpoint and pass access/refresh token pairs via HttpOnly cookies * perf(update user): get user id exclusively from route * docs: add sequence flow diagrams for different scenarios * perf(refresh-token): reduce the no generated refresh tokens Generate refresh tokens only for specific scenarios, not for every kind of request: - valid refresh/access token pair provided - refresh/access token pair not already used to generate a new access token - refresh token not expired - access token expired - etc. * build(docker): add dockerfile for quetzalcoatl microservice * perf(refresh token): add trigger for deleting stale tokens * perf(refresh token): replace int with timespan for token lifetimes * perf(config): use IOptions * feat: add health check endpoint * build(docker): add docker-compose * perf(docker): assure that db is started before the api * Test merging strategy (#39)
WarriorsSami
added a commit
that referenced
this pull request
Oct 31, 2024
* feat: add init project structure * feat(identity): scaffold db for default identity * docs(auth): add architecture diagram * feat(config): add jwt config * perf: move endpoints to separate proj than di * feat: add migrations for default identity * feat(infr): add new migrations for custom user * feat(domain): add user entity and jwt config * perf(di): add custom user to identity auth * feat(test): scaffold integration tests * feat(app): implement handlers required by auth -CreateUser -ValidateUserCredentials -GenerateJwtToken * refactor(test): split test proj between integration and unit * perf(mapper): use automapper for register endpoint * perf(app): organize handlers based on vertical slice * feat(test): add fixture for test containers * feat(test): create integration test for register endpoint * perf(api): refactor endpoints to improve readability * fix(test): remove sqlconnection error and speed up test containers - Declare additional IServiceCollection extension methods for ensuring db created and removing already existing DbContextOptions<AppDbContext> (not AppDbContext!!) - Use the latest MsSql Docker Image * feat(di): add default swagger support * test: remove unit tests proj As Fastendpoints Commands are hard to unit test outside of an Endpoint, it is enough to have only Integration tests to cover everything * test(login): add integration test for login * style: rename username prop and move jwt mapper here * docs(swagger): add swagger support * feat(users): scaffold CRUD users api * style(auth): move validators to separate files * feat(users): add get all endpoint * test(users): add integration tests for get all endpoint * feat(users): add get user endpoint * test(users): add integration test for get user endpoint * style(tests): add regions to enhance readability * style: replace local usings with global usings * perf(user): add guid as pk for application user * feat(update): create update user endpoint * test(update): create integration tests for update endpoint * misc: update .gitignore * perf(user manager): use predefined methods instead of LINQ for finding by id * feat(api): create delete endpoint * style: add regions and update global usings * test(delete): create integration tests for delete endpoint * docs(swagger): add summaries for Identity CRUD endpoints * style(binding): remove redundant bind from attribute for models Id * perf(update); enable request fields to be nullable In order to allow partial update for user profile (similar to patch), enable validation/mapping rules only when a certain request field is provided * style: add global usings * test(update): add integration test for partial update * style: format code * style: adjust namespace for jwt config * feat(user): add support for fullname and bio fields * test(user): update integration tests to support fullname/bio fields * feat(image): add support for user profile image * test(register): update integration tests to support profile image * perf(picture): enable support for picture in login endpoint Enable eager loading in db context for profile picture in order to be able to retrieve the image id for the download URL * feat(build): add migrations for profile picture * test(login): update login tests for picture Replace register endpoint invocation with the user manager directly in order to mitigate the tight coupling between tests and the QAPI * feat(picture): update the rest of endpoints to support profile picture * test(picture): update the rest of tests to support profile picture * feat(image): add get image endpoint * test(image): add integration test for get image endpoint * perf(image): create image repository * style: format code * perf: add argument null guards to ctors * feat(logger): configure serilog * misc: add Logs to .gitignore * perf: add logging to endpoints * fix(logger): replace bootstrap logger with default Bug related to integration tests fails * feat(update): add permission guard to endpoint Allow users to edit only their own profile, otherwise throw unauthorized * test: add integration test for self update permission * perf(update): replace unauthorized with forbidden * perf: use default identity role * perf: set model fields as init-only * feat: add seed data for roles and admin user * test(get all): remove users already added by seeding * perf(delete): restrict access to endpoint using roles from jwt * test(delete): add test for non-admin user * perf(image): enable profile image to be nullable * feat(api): add support for refresh tokens Add RefreshToken endpoint and pass access/refresh token pairs via HttpOnly cookies * perf(update user): get user id exclusively from route * docs: add sequence flow diagrams for different scenarios * perf(refresh-token): reduce the no generated refresh tokens Generate refresh tokens only for specific scenarios, not for every kind of request: - valid refresh/access token pair provided - refresh/access token pair not already used to generate a new access token - refresh token not expired - access token expired - etc. * build(docker): add dockerfile for quetzalcoatl microservice * perf(refresh token): add trigger for deleting stale tokens * perf(refresh token): replace int with timespan for token lifetimes * perf(config): use IOptions * feat: add health check endpoint * build(docker): add docker-compose * perf(docker): assure that db is started before the api * Test merging strategy (#39) * fix(quetzalcoatl): remove obsolete role enum (#41) * fix(quetzalcoatl): remove redundant compose file (#44) * fix(quetzalcoatl): remove obsolete role enum * fix(quetzalcoatl): remove redundant compose file * Quetzalcoatl auth/psql migration (#47) * fix(quetzalcoatl): remove obsolete role enum * fix(quetzalcoatl): remove redundant compose file * fix(ci): use correct regex to extract version from tag
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.