feature/configurable-logging-causes #3663
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…efault til 480 hvis ikke satt (som før, men legger nå på " (truncated)..." i tillegg). - Kan nå legge til causes på stack trace. Default til false hvis ikke satt (som før). - Kan nå filtrere på elementer i stack trace. Default til alle hvis ikke satt (som før).
krharum
pushed a commit
that referenced
this pull request
Oct 28, 2024
- Kan nå konfigurere eller slå av lengde før stack trace trunkeres. Default til 480 hvis ikke satt (som før, men legger nå på " (truncated)..." i tillegg). - Kan nå legge til causes på stack trace. Default til false hvis ikke satt (som før). - Kan nå filtrere på elementer i stack trace. Default til alle hvis ikke satt (som før).
krharum
added a commit
that referenced
this pull request
Oct 29, 2024
* Refactor AnsettelseService to remove manual threading. #deploy-levende-arbeidsforhold-ansettelse Removed the explicit use of Threads and handling interruptions. Streamlined the service execution and added error logging for better maintainability and debugging. * Increase pending acquire timeout in AaregConsumer #deploy-levende-arbeidsforhold-ansettelse Extended the pending acquire timeout from 300 seconds to 3000 seconds for AaregConsumer connections. This change aims to reduce connection acquisition failures during high latency periods. * Increase proxy timeouts in PDL Proxy config #deploy-pdl-proxy Extended the `nginx.ingress.kubernetes.io/proxy-read-timeout` and `proxy-send-timeout` annotations from 600 to 2400 seconds. This change aims to accommodate longer processing times required by the application. * Introduce delay in processing elements #deploy-levende-arbeidsforhold-ansettelse Added a delay of one minute between processing each element within the 'opprettAnsettelse' method to prevent overwhelming external services. This change aims to enhance system stability and prevent any rate-limiting issues. * This code diff introduces a significant change by migrating the application from a traditional Spring MVC architecture to a reactive Spring WebFlux architecture. This change is evident in the following modifications: **1. Reactive Dependencies:** - Removal of `spring-boot-starter-data-jpa` and addition of `spring-boot-starter-data-r2dbc` indicate a shift from blocking JPA to reactive R2DBC for database interactions. - Inclusion of `org.postgresql:r2dbc-postgresql` and `io.r2dbc:r2dbc-h2` provides drivers for reactive database connections. **2. Reactive Repositories:** - Repositories like `ParameterRepository`, `AnsettelseLoggRepository`, and `LoggRepository` now extend reactive interfaces like `ReactiveCrudRepository` and `ReactiveSortingRepository`. - Methods in these repositories now return reactive types like `Flux` and `Mono` instead of `List` and `Optional`. **3. Reactive Services:** - Service methods in `LoggService`, `ArbeidsforholdService`, `KodeverkService`, `AnsettelseLoggService`, and `ParameterService` are updated to return reactive types (`Flux`, `Mono`). - Usage of reactive operators like `flatMap`, `map`, and `collectList` is introduced for asynchronous data processing. **4. Reactive Controllers:** - Controllers like `ParameterController` and `LoggController` now return reactive types (`Flux`, `Mono`) from their endpoint methods. **5. Security Configuration:** - The `SecurityConfig` class is updated to use `EnableWebFluxSecurity` and `EnableReactiveMethodSecurity` annotations, indicating a shift to reactive security configurations. - The `springSecurityFilterChain` method now configures a `SecurityWebFilterChain` for reactive web requests. **6. Other Changes:** - Introduction of `NavHeaders` class for managing custom headers. - Update to OpenApiConfig to support reactive endpoints. - Removal of unused dependencies and code related to the previous Spring MVC architecture. **Impact:** This migration to a reactive architecture brings several benefits: - **Improved Performance and Scalability:** Reactive applications can handle more requests with fewer resources compared to traditional blocking applications. - **Non-Blocking Operations:** Asynchronous operations prevent threads from being blocked, leading to better resource utilization. - **Enhanced Responsiveness:** Reactive applications can respond to user requests faster, even under heavy load. **Considerations:** - Developers need to be familiar with reactive programming concepts and libraries like Reactor to work with this updated codebase. - Testing strategies need to be adapted for reactive components. - Existing code that interacts with the application might need adjustments to handle reactive types. This migration represents a significant architectural shift that can lead to a more performant and scalable application. * Add error handling and logging updates Included a new JsonNode for errors in PdlPersonDTO and added Jackson codecs in PdlConsumer. Enhanced logging in SokPersonCommand for debugging purposes. * Update and optimize logging repository and services #deploy-levende-arbeidsfoehold-ansettelse Refactor LoggRepository to use Mono for count and remove unnecessary vault dependencies. Added delay in ArbeidsforholdService to manage requests better and set restart policy in docker-compose. Introduced PageableHandlerMethodArgumentResolver for custom pageable handling and adjusted entity timestamp types. * Deploy #deploy-levende-arbeidsforhold-ansettelse * Switch to R2DBC for datasource configuration #deploy-testnav-levende-arbeidsforhold-ansettelse Updated the datasource configuration to use the R2DBC URL instead of the JDBC URL for PostgreSQL. Also, included username and password properties to align with the new configuration requirements. This change aims to improve non-blocking database interaction. * Deploy #deploy-levende-arbeidsforhold-ansettelse * Update logger encoder class for prod profile #deploy-levende-arbeidsforhold-ansettelse Changed the encoder class in the prod profile from `no.nav.testnav.libs.servletcore.logging.TestnavLogbackEncoder` to `no.nav.testnav.libs.reactivecore.logging.TestnavLogbackEncoder`. This ensures logs are managed by the appropriate logging framework for reactive applications. * Add r2dbc datasource configuration to application-prod.yml #deploy-levende-arbeidsforhold-ansettelse This change includes the addition of `r2dbc` configuration parameters for URL, username, and password in the `application-prod.yml` file. This ensures that the application has the necessary parameters to connect to the database using R2DBC. * Switch datasource URL from r2dbc to jdbc in prod config. #deploy-levende-arbeidsforhold-ansettelse Updated the datasource URL in the production configuration file to use JDBC instead of R2DBC. This change ensures compatibility and proper connectivity for database operations in the production environment. * Add R2DBC username and password to production config #deploy-levende-arbeidsforhold-ansettelse Updated the `application-prod.yml` to include necessary credentials for the R2DBC connection. This change ensures proper authentication and secure access to the PostgreSQL database. * Add environment variables from secret #deploy-levende-arbeidsforhold-ansettelse This commit updates the config.yml file to include environment variables sourced from the specified secret. This change is necessary for managing sensitive information securely in the application. It ensures that the application can access necessary credentials without hardcoding them into the configuration file. * Update datasource to r2dbc and reorganize configuration #deploy-levende-arbeidsforhold-ansettelse Replaced "datasource" with "r2dbc" for consistency and updated Flyway configurations to separate DB and R2DBC settings. These changes improve clarity and maintenance of configuration files. * Switch spring datasource to flyway in test config #deploy-levende-arbeidsforhold-ansettelse Updated the application.yaml to replace the spring datasource configuration with a flyway configuration for the test environment. This change is intended to streamline database migrations during testing. * Add SSL certificate to database URLs in production config #deploy-levende-arbeidsforhold-ansettelse Updated the database URLs in the production configuration file to include the SSL certificate parameter. This change ensures secure communication with the database using SSL certificates. * Update database URLs in production config #deploy-levende-arbeidsforhold-ansettelse Simplified the database URLs in `application-prod.yml` by using a single environment variable for each URL. This change reduces redundancy and simplifies the configuration management. * Update Flyway URL to use JDBC environment variable #deploy-levende-arbeidsforhold-ansettelse The Flyway URL configuration in the production YAML file now references the correct JDBC URL environment variable. This aligns the configuration with the expected environment setup and fixes potential issues with database migrations. * Update database configuration for production environment #deploy-levende-arbeidsforhold-ansettelse Modified the R2DBC URL to include host, port, and database name, and added username and password fields. This ensures all necessary connection details are properly configured for production. * Enable R2DBC repositories and update DB config #deploy-levende-arbeidsforhold-ansettelse Enabled R2DBC repositories in the Spring data configuration. Adjusted the R2DBC URL by removing the database name from the URL and added a separate 'name' field for the database. This improves clarity and consistency in database configuration settings. * Enable WebFlux and remove unnecessary R2DBC and envFrom settings #deploy-levende-arbeidsforhold-ansettelse Enabled WebFlux in the application configuration for reactive web support. Removed redundant R2DBC repository settings and obsolete environment variables, streamlining the configuration files. * Update dependencies in build and settings gradle files #deploy-levende-arbeidsforhold-ansettelse Removed unused 'database' and 'HikariCP' dependencies. Added 'spring-boot-starter-security' to enhance security features. These changes streamline the project dependencies and improve maintenance. * Switch to OAuth2 resource server dependency #deploy-levende-arbeidsforhold-ansettelse This change replaces the OAuth2 client dependency with the OAuth2 resource server dependency in build.gradle. This adjustment is necessary to align with the security requirements for handling resource server functionalities in the application. * Add PostgreSQL connection factory for production profile #deploy-levende-arbeidsforhold-ansettelse Introduced a new bean to configure a PostgreSQL connection factory when the 'prod' profile is active. This setup utilizes environment variables for connection details. * Add profile annotation and update health check paths #deploy-levende-arbeidsforhold-ansettelse Moved the @Profile("dev") annotation to the class level to streamline configuration. Updated liveness and readiness probe paths and delays in config.yml for better alignment with the new health endpoints. * Remove production database config and refine R2DBC setup #deploy-levende-arbeidsforhold-ansettelse Eliminated hardcoded production database connection settings. Enabled auto-configuration for R2DBC auditing and repositories. Adjusted application-dev.yml for cleaner configuration. * Add database connection pool validation query #deploy-levende-arbeidsforhold-ansettelse Added a validation query to the database connection pool configuration in the application-prod.yml file. This ensures the connection's health is checked by running a simple query, enhancing reliability. * Reorganize dependencies in build.gradle #deploy-levende-arbeidsforhold-ansettelse Move r2dbc-postgresql and r2dbc-h2 implementations for better structure and readability. Adjust runtimeOnly for h2 dependency to ensure proper runtime behavior. * Remove H2 console and runtime dependency #deploy-levende-arbeidsforhold-ansettelse Disabled the H2 console and removed the corresponding runtime dependency from build.gradle. This change simplifies the configuration and reduces unnecessary dependencies. * Refactor R2DBC configuration for dynamic database setup #deploy-levende-arbeidsforhold-ansettelse Updated R2DBC settings in YAML files to support dynamic database configuration. Introduced ConnectionFactory bean in ApplicationConfig to dynamically configure connection-related parameters. Added new configuration settings for both production and development environments. * Add protocol support for R2DBC configuration #deploy-levende-arbeidsforhold-ansettelse Introduced protocol field in R2DBC configuration to enhance flexibility. Adjusted configurations to conditionally include host and protocol options if they are present. Updated application configuration files to use the new protocol setting. * Switch to Spring's flyway and r2dbc configuration Add protocol support for R2DBC configuration #deploy-levende-arbeidsforhold-ansettelse Replaced custom R2DBC configurations with Spring's built-in settings in application YAML files and annotated out manual configuration in Java. This streamlines database connectivity and simplifies future maintenance. * Update database connection URL in application-prod.yml #deploy-levende-arbeidsforhold-ansettelse Changed the R2DBC URL to include specific PostgreSQL connection parameters. This ensures the database connection is properly configured with the required host, port, database name, and SSL settings. * Update DB URL to use environment variable placeholders #deploy-levende-arbeidsforhold-ansettelse Replaced hardcoded SSL path values in the database URL with corresponding environment variable placeholders. This change enhances security and flexibility by centralizing the configuration. * Set root logging level to DEBUG in logback configuration #deploy-levende-arbeidsforhold-ansettelse This change modifies the root logging level from INFO to DEBUG in the `logback-spring.xml` file. This adjustment enables more detailed logging output for better debugging during development or troubleshooting. * Change root log level to INFO Updated the root logging level from DEBUG to INFO in logback-spring.xml to reduce log verbosity in production. This will help in focusing on important informational messages and errors. * Update database configuration in application-prod.yml #deploy-levende-arbeidsforhold-ansettelse Refactored the R2DBC URL structure, separating credentials and SSL properties for clarity. This change improves maintainability and aligns with best practices for secure database connections. * Update database URL and comment out SSL properties #deploy-levende-arbeidsforhold-ansettelse Changed database URL to a fixed IP address and commented out the SSL properties. This modification may impact secure connections and should be reviewed prior to production deployment. * Remove connection pool configuration #deploy-levende-arbeidsforhold-ansettelse This change removes the connection pooling from the R2DBC URL in the production application configuration file. It ensures a direct connection to the PostgreSQL database without using a pool. * Update database connection settings in production config Remove connection pool configuration #deploy-levende-arbeidsforhold-ansettelse Changed the R2DBC URL to include connection pooling and enabled SSL properties with environment-specific variables. This enhances security and connection management for the production database. * Enable SSL debugging and enforce SSL verification #deploy-levende-arbeidsforhold-ansettelse Added SSL debugging options to JAVA_OPTS for better diagnostics. Changed the SSL mode to VERIFY_FULL in the production configuration to enforce stricter SSL verification. * Update SSL properties in application-prod.yml #deploy-levende-arbeidsforhold-ansettelse Replaced hard-coded SSL values with environment variables for better configuration flexibility. Adjusted `sslmode` and `sslkey` properties to use respective environment variables. * Disable Flyway database migration in production #deploy-levende-arbeidsforhold-ansettelse Commented out Flyway URL, username, and password configuration in application-prod.yml. Added a new property to disable Flyway migrations in the production environment. * Constructing JDBC URL in same manner as R2dbc URL, for testing. * Fixed typo in Flyway URL. #deploy-levende-arbeidsforhold-ansettelse * Added dependency on r2dbc-postgresql. #deploy-levende-arbeidsforhold-ansettelse * Moved from pool defined in URL (handled by R2DBC) and pool defined in config (handled by Spring). #deploy-levende-arbeidsforhold-ansettelse * - Changed dependencies based on a new Spring Init module w/R2DBC and PostgreSQL. - Added full URL with params for R2DBC. - Removed pool config. - Commented out ShortenedThrowableConverter (for now). #deploy-levende-arbeidsforhold-ansettelse * - Setting spring.r2dbc.pool.enabled=false (for now). #deploy-levende-arbeidsforhold-ansettelse * - Setting spring.data.r2dbc.repositories.enabled=true (for now). #deploy-levende-arbeidsforhold-ansettelse * - Attempting to remove H2 (for now). - Removed pool and repository config (no effect). - Stack trace no longer cut at 480 chars (for now). - Enabled debug mode to check bean resolution. #deploy-levende-arbeidsforhold-ansettelse * - Disabled test (for now; caused by disabling H2). #deploy-levende-arbeidsforhold-ansettelse * - Attempting to manually create a R2dbcEntityTemplate bean. #deploy-levende-arbeidsforhold-ansettelse * - Turning off debug. - Attempting to name bean. #deploy-levende-arbeidsforhold-ansettelse * Tester bruk av com.google.cloud.sql:r2dbc-postgres. #deploy-levende-arbeidsforhold-service * Endret til R2dbcRepository (som extends tidligere). * - Prøver endret format på filreferanser. - Fjerner com.google.cloud.sql:cloud-sql-connector-r2dbc-postgres. #deploy-levende-arbeidsforhold-ansettelse * - Lager nå en eksplisitt ConnectionFactory bean basert på konfigurert URL i spring.r2dbc.url. - Flytter @EnableR2dbc* til @SpringBootApplication, for oversikt. - Retter config for test (annet format URL for R2dbc mot H2). - Context-testen dobbeltsjekker at vi får en R2dbcEntityTemplate bean (fra AbstractR2dbcConfiguration). - Lagt til io.r2dbc:r2dbc-h2 som test dependency. - Erstattet org.postgresql:r2dbc-postgresql med io.r2dbc:r2dbc-postgresql som runtime dependency. #deploy-levende-arbeidsforhold-ansettelse * - Lager nå en eksplisitt bean R2dbcEntityTemplate. #deploy-levende-arbeidsforhold-ansettelse * - Endret fra io.r2dbc:r2dbc-postgresql til org.postgresql:r2dbc-postgresql, som faktisk virker. - Fjernet file:// prefix etter å ha verifisert at kode slår opp og leser inn fra filer på angitt navn. - Fjernet R2dbcConfiguration, siden den ikke trengs for verken prod eller test med org.postgresql:r2dbc-postgresql. #deploy-levende-arbeidsforhold-ansettelse * - Utvider TestnavLogbackEncoder til å ta config maxStackTraceLength (default som før). - Endrer logging midlertidig pga. testing. #deploy-levende-arbeidsforhold-ansettelse * - Rettet logging, med unlimited stack trace. #deploy-levende-arbeidsforhold-ansettelse * - Logger nå causes rekursivt om ønskelig (default off, som før). #deploy-levende-arbeidsforhold-ansettelse * Logger kun no.nav.testnav i stack trace. #deploy-levende-arbeidsforhold-ansettelse * - Prøver PK8 private key (rammeverket kan ikke lese format på PEM). - Slår av stack trace elements i loggen. #deploy-levende-arbeidsforhold-ansettelse * - Lager URL ut fra config. #deploy-levende-arbeidsforhold-ansettelse * - Fjerner sslCert og sslKey. #deploy-levende-arbeidsforhold-ansettelse * - Legger til sslCert uten sslKey. #deploy-levende-arbeidsforhold-ansettelse * - Tester uten ssl i properties, men med både sslCert og sslKey og tomt sslPassword. - Lagt til test på om sslCert-fil faktisk finnes. #deploy-levende-arbeidsforhold-ansettelse * - Health actuator viser detaljer. - Fjerner sslKey til fordel for testkode. #deploy-levende-arbeidsforhold-ansettelse * - Rettet manglende config for test. #deploy-levende-arbeidsforhold-ansettelse * - Alle spring.r2dbc.properties nå som URL-parametre. - Tester (for syns skyld) manuel load av PK8. #deploy-levende-arbeidsforhold-ansettelse * - La til manglende config for test (ikke brukt). #deploy-levende-arbeidsforhold-ansettelse * - Lagt til user og password i URL. #deploy-levende-arbeidsforhold-ansettelse * - Lagt til test-config for sjekk av filer. #deploy-levende-arbeidsforhold-ansettelse * - Fjernet feil dependency. #deploy-levende-arbeidsforhold-ansettelse * - Lagt til init-script som konverterer til PKCS#8-format. -Reverterer config til å bruke spring.r2dbc.properties, for lesbarhet. - Flytter application wide config til applikasjonsklassen og fjerner testkode i ApplicationConfig. #deploy-levende-arbeidsforhold-ansettelse * - Slår av SSL debugging. - Opprydding. #deploy-levende-arbeidsforhold-ansettelse * - Cleanup imports. - Presisering av format ved konvertering. * feature/configurable-logging-causes (#3663) - Kan nå konfigurere eller slå av lengde før stack trace trunkeres. Default til 480 hvis ikke satt (som før, men legger nå på " (truncated)..." i tillegg). - Kan nå legge til causes på stack trace. Default til false hvis ikke satt (som før). - Kan nå filtrere på elementer i stack trace. Default til alle hvis ikke satt (som før). * Introduce ArbeidsforholdResponseDTO for detailed responses #deploy-levende-arbeidsforhold-ansettelse Replaces HttpStatusCode with ArbeidsforholdResponseDTO to capture both status and detailed error messages. Refactors all related methods and classes to support this change, improving error handling and logging. * - Mindre framtidig forvirring; vi konverterer til en PKCS#8 (pk8) private key lagret på formatet PEM. * Remove sort order from API call in NyansettelserPage.tsx #deploy-test-front #deploy-frontend The sort order parameter 'id,DESC' was removed from the useLevendeArbeidsforholdLogg API call. This change simplifies the function call and relies on the API's default sorting behavior. * Implement retry logic for ansettPerson method #deploy-levende-arbeidsforhold-ansettelse Adds retry mechanism to handle failures when attempting to create new employment records, with up to 3 retries. Introduces AtomicInteger for tracking retry attempts and refactors the logic to ensure robust error handling and logging. --------- Co-authored-by: Cato Olsen <[email protected]> Co-authored-by: Cato Olsen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Utvider TestnavLogbackEncoder til å inkludere causes, med konfigurerbar filtrering på lengde og innhold i stack trace. Defaults er å oppføre seg som før, siden det p.t. påvirker 19 apps og 24 proxies.