edge

fix: garbage collection of merge-request refs when webhook mode (#859)

* feat: add merge request handle to garbage collect closed mr refs

* refactor: simplify the code

we already have a deleteRef function

* chore: add debug log to track received events

* fix: block should not end with a whitespace

v0.5.8

What's Changed

• fix(devenv): Fix devenv rule in makefile. by @audig in #822
• Add gitlab-health-url flag and envvar by @Thor77 in #800
• fix project pull webhook args by @fitz7 in #828
• go.mod: updated most dependencies

New Contributors

• @audig made their first contribution in #822
• @Thor77 made their first contribution in #800
• @fitz7 made their first contribution in #828

Full Changelog: v0.5.7...v0.5.8

v0.5.7

What's Changed

• fix environment id hyperlink NaN by @toby181 in #776
• webhooks: handle ref and tag deletion events by @sysedwinistrator in #794
• webhooks: only pull project for which request was received by @sysedwinistrator in #793
• ratelimit: redis should retry if allowed requests exceeded by @bkylerussell in #789
• chore: dependencies, tooling and go updates by @mvisonneau in #799

New Contributors

• @toby181 made their first contribution in #776
• @sysedwinistrator made their first contribution in #794
• @bkylerussell made their first contribution in #789

Full Changelog: v0.5.6...v0.5.7

v0.5.6

What's Changed

• fix default branch regexp by @gnomus in #665
• Add test cases metrics from pipeline reports by @karpanin in #675
• bump go-gitlab version, bump golang version for make dev-env by @jasonwliu in #670
• Combine test report metrics with childrens reports by @jasonwliu in #672
• Fix misformed url to Gitlab pipeline by @fredsnap in #650
• Update go-gitlab to fix test reports by @clawoflight in #686
• Update GitLab logo to new branding in Grafana dashboard quickstart by @dnsmichi in #694
• Add tag_list label to metrics by @mpetke in #499
• pipelines: store the source of the pipeline as well by @mathstuf in #723
• feat: failure reason to job status by @strpc in #718
• build(deps): bump github.com/xanzy/go-gitlab from 0.92.3 to 0.94.0 by @dependabot in #743
• build(deps): bump github.com/charmbracelet/lipgloss from 0.8.0 to 0.9.1 by @dependabot in #735
• gitlab/client: guard against nil responses by @mathstuf in #729
• build(deps): bump golang.org/x/net from 0.15.0 to 0.17.0 by @dependabot in #732
• build(deps): bump github.com/alicebob/miniredis/v2 from 2.23.0 to 2.31.0 by @dependabot in #734
• build(deps): bump github.com/google/uuid from 1.3.1 to 1.4.0 by @dependabot in #737
• build(deps): bump golang.org/x/time from 0.3.0 to 0.4.0 by @dependabot in #741
• build(deps): bump github.com/go-playground/validator/v10 from 10.15.4 to 10.16.0 by @dependabot in #742
• build(deps): bump github.com/redis/go-redis/v9 from 9.2.1 to 9.3.0 by @dependabot in #740
• build(deps): bump github.com/vmihailenco/msgpack/v5 from 5.4.0 to 5.4.1 by @dependabot in #739
• build(deps): bump google.golang.org/grpc from 1.58.2 to 1.59.0 by @dependabot in #736
• feat: use keyset pagination for retrieving project CI jobs by @stanhu in #744
• fix: use keyset pagination only when it is supported by @stanhu in #755

New Contributors

• @gnomus made their first contribution in #665
• @karpanin made their first contribution in #675
• @fredsnap made their first contribution in #650
• @clawoflight made their first contribution in #686
• @mpetke made their first contribution in #499
• @mathstuf made their first contribution in #723
• @strpc made their first contribution in #718
• @stanhu made their first contribution in #744

Full Changelog: v0.5.5...v0.5.6

v0.5.5

Added

• new metrics:
  • gitlab_ci_pipeline_test_report_total_time -> Duration in seconds of all the tests in the most recently finished pipeline
  • gitlab_ci_pipeline_test_report_total_count -> Number of total tests in the most recently finished pipeline
  • gitlab_ci_pipeline_test_report_success_count -> Number of successful tests in the most recently finished pipeline
  • gitlab_ci_pipeline_test_report_failed_count -> Number of failed tests in the most recently finished pipeline
  • gitlab_ci_pipeline_test_report_skipped_count -> Number of skipped tests in the most recently finished pipeline
  • gitlab_ci_pipeline_test_report_error_count -> Number of errored tests in the most recently finished pipeline
  • gitlab_ci_pipeline_test_suite_total_time -> Duration in seconds for the test suite
  • gitlab_ci_pipeline_test_suite_total_count -> Number of total tests for the test suite
  • gitlab_ci_pipeline_test_suite_success_count -> Number of successful tests for the test suite
  • gitlab_ci_pipeline_test_suite_failed_count -> Number of failed tests for the test suite
  • gitlab_ci_pipeline_test_suite_skipped_count -> Number of skipped tests for the test suite
  • gitlab_ci_pipeline_test_suite_error_count -> Duration in errored tests for the test suite
• new configuration parameter: gitlab.burstable_requests_per_second, introducing a burstable amount of API RPS
• new configuration parameter: gitlab.maximum_jobs_queue_size, controlling the queue buffer size

Changed

• Upgraded golang to v1.20
• Upgraded most dependencies to their latest versions
• Reduced the amount of data being pulled from the project list API calls

v0.5.4

Added

• Kickstarted tracing support through opentelemetry implementation for most of the network calls
• Now passing a context.Context to most functional calls
• Aggregated already used linters and added new ones through the implementation of golangci
• Release .apk packages for Alpine linux
• Added man pages and autocompletion scripts (bash & zsh) to .apk, .deb, .rpm & homebrew packages
• Release "fat" binaries (arm64 + amd64 combined) for MacOS under _all suffix

Changed

• Fixed a config issue preventing the arm deb/rpm packages to be released correctly
• Upgraded golang to v1.19
• Upgraded most dependencies to their lastest versions
• Fixed child pipeline jobs not found whilst looking up through bridges (#345)
• gitlab_ci_pipeline_job_queued_duration_seconds & gitlab_ci_pipeline_queued_duration_seconds will now be leveraging the value returned through the GitLab API instead of computing it with (startedAt - createdAt)
• Refactored the RPC layer used for CLI monitoring with gRPC

v0.5.3

Added

• linux/arm/v6 & linux/arm/v7 binary & container image releases
• quay.io container image releases
• New internal metrics about exporter's health:
  • gcpe_gitlab_api_requests_remaining - GitLab API requests remaining in the API Limit
  • gcpe_gitlab_api_requests_limit - GitLab API requests available in the API Limit

Changed

• Fixed an issue when running in cluster mode where tasks could hang when the exporter restarted.
• Fixed a bug in some cases where pagination headers are not returned from GitLab's API
• Upgraded most dependencies to their latest versions
• fixed json syntax error in webhook error body
• dashboards: fixed owner multiselect filters
• fixed a bug on gitlab_ci_pipeline_run_count being incremented multiple times

v0.5.2

Changed

• Updated default metrics garbage collection intervals from 5 to 10m
• monitor: correctly sanitize the config output
• fixed an issue introduced in v0.5.1 causing the exporter to fail when the monitoring listener address was not defined
• fixed a bug that could cause nil pointer deferences on readiness checks
• Updated golang to 1.17

v0.5.1

Added

• New monitoring CLI to retrieve information about the exporter
• New internal metrics about exporter's health:
  • gcpe_currently_queued_tasks_count - Number of tasks in the queue
  • gcpe_environments_count - Number of GitLab environments being exported
  • gcpe_executed_tasks_count - Number of tasks executed
  • gcpe_gitlab_api_requests_count - GitLab API requests count
  • gcpe_metrics_count - Number of GitLab pipelines metrics being exported
  • gcpe_projects_count - Number of GitLab projects being exported
  • gcpe_refs_count - Number of GitLab refs being exported

Changed

• fixed a goroutines/memory leak introduced in 0.5.0 which can occur when attempting to process more tasks than the ratelimit permits
• fixed a bug causing the refreshing of tag based jobs to not match any API results, generating lots of unnecessary requests
• webhooks: added more details when processing wildcards
• examples/quickstart: updated prometheus and grafana versions
• updated most libraries to their most recent versions

v0.5.0

BREAKING CHANGES

• The configuration syntax has evolved, you can refer to the updated version of the doc

  • pull.maximum_gitlab_api_requests_per_second has moved to gitlab.maximum_requests_per_second
  • project_defaults.pull.refs.* has been redone in a hopefully more comprehensible fashion
    • project_defaults.pull.refs.(branches|tags|merge_requests).* parameters can now be used to have a finer granularity
      on the management of per-ref-kind settings
    • project_defaults.pull.refs.from.(pipelines|merge_requests) is not natively managed as part of the per-ref-kind specific parameters
  • project_defaults.pull.environments.name_regexp has moved to project_defaults.pull.environments.regexp
  • project_defaults.pull.environments.tags_regexp was removed to avoid confusion

• Logging configuration now has to be done as part of the config file instead of CLI flags:

log:
  level: info
  format: text

• By default, when exporting metrics for environments, stopped ones will not be considered any more.

Added

• New metric gitlab_ci_pipeline_queued_duration_seconds
• New metric gitlab_ci_pipeline_job_queued_duration_seconds
• (en|dis)able pulling branches / tags / merge_requests on a global or per-project basis
• Choose to export metrics for only the most 'n' recently updated refs pipelines
• Choose to export metrics for refs with pipelines 'updated in the last x seconds'
• Choose to export metrics for deleted branches or tags
• Choose to export metrics for available environments only

Changed

• Bumped all dependencies
• Enhanced the function signatures for the ref objects management
• Fixed a bug causing the jobs metrics pull to fail on ref.Kind=RefKindMergeRequest
• Fixed a bug causing the environments to be garbage collected on every run
• Fixed the error handling when comparing 2 refs which resulted into nil pointer dereferences
• Fixed the pulling of merge-request based pipelines
• Fixed unit tests on windows
• Replaced custom config mangement implementation with creasty/defaults and go-playground/validator
• Rewrote the non-OOP approach used so far for the controller