Redesign DB Draft and Proposed Pushing

[rendre-greyling]

Description

The previous implementation of pushing draft and proposed package revisions mimicked the CR cache by pushing a draft and proposed package revision after every update. This resulted in a Package Revision being pushed at minimum 4 times from creation to Published state. Many further issues were present such as race conditions, unwanted deletions of Package Revisions and increased load on the external Git instance. The design was quite terrible and unstable - I can say this since I implemented the feature :)

This new implementation completely removes this logic and moves the push logic for draft and proposed Package Revisions to the sync process in the DB cache (which is triggered by the Repository controller).

• When a Package Revision is created, it is not pushed to the external git. If it is subsequently updated, proposed and approved before the next sync runs, then only the approved tag for the approved Package Revision will be present in the external git (the same as without using the push drafts to git feature).
• If a Package Revision is created, but not approved and the next sync runs, the Package Revision will be pushed to the external git as a branch.
• If said Package Revision is Proposed and the next sync runs, the Package Revision is pushed to the external git as a proposed branch (the draft branch is deleted). If then approved, the proposed branch is deleted, and the approved tag is created.
• A proposed branch does not require there to have been a draft branch before it and an approval does not require there to have been a draft or proposed branch. It will delete either one which is present when creating the tag.

Reasoning of this change:

• Preserves the ability to only have to back up the external git data as draft and proposed Package Revisions will be reflected in the external git and the DB Cache can be rebuilt from this.
• Significantly decreased load to the external Git server during automated hydration operations since in most cases a Package Revision will be created and approved in a short span of time before a sync runs.
• Increased speed of Hydration, less pushes = less hydration time and less load on external git server.
• It has been observed in some cases that deleting draft and proposed branches can be expensive and take significant amount of time in the external git, so eliminating this is crucial.

---

Related Issue(s)

Related to: #1143

---

Usage

Set the db-push-drafts-to-git flag in the repository controller and porch-server to true.

---

Type of Change

• Bug fix
• New feature
• Enhancement
• Refactor
• Documentation
• Tests
• Other: ________

---

Checklist

• Code follows project style guidelines
• Self-reviewed changes
• Tests added/updated
• Documentation added/updated
• All tests and gating checks pass

---

AI Disclosure

• I have used AI in the creation of this PR.

composer 2.5 for UT generation and assistance with e2e tests.

[netlify]

✅ Deploy Preview for kpt-porch ready!

Name	Link
🔨 Latest commit	9ac3edc
🔍 Latest deploy log	https://app.netlify.com/projects/kpt-porch/deploys/6a7cd40e4e91940008bc82ea
😎 Deploy Preview	https://deploy-preview-1146--kpt-porch.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Copilot]

Pull request overview

Refactors the DB-cache “push drafts/proposed to Git” behavior so draft/proposed revisions are pushed during repository sync (triggered by the Repository controller), reducing repeated pushes and avoiding prior race/deletion behaviors. It also adds DB markers and expanded test coverage (unit + E2E) to validate the new sync-driven push semantics.

Changes:

• Move draft/proposed push-to-git behavior to DB repo sync, introducing last_pushed_db_updated to detect when DB content needs pushing.
• Replace the old engine push implementation with DB-cache-specific push helpers for published and draft/proposed revisions.
• Add extensive E2E coverage for DB↔Git sync edge cases and update deployment/scripts to wire the controller flag.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/e2e/suiteutils/suite.go	Adds configurable Gitea credentials/host to E2E suite initialization.
test/e2e/suiteutils/suite_utils.go	Adds filtered PR lookup helpers and repo sync trigger/wait helpers for tests.
test/e2e/suiteutils/gitea_test_utils.go	Expands Gitea test utilities (repo CRUD, branch/tag helpers, content commits).
test/e2e/api/db_git_sync_test.go	New E2E suite covering DB-cache sync-driven draft/proposed/published Git behavior.
scripts/create-deployment-blueprint.sh	Enables push-drafts-to-git flags for both porch-server and controllers.
pkg/externalrepo/fake/repository.go	Makes fake UpdatePackageRevision return a usable draft object.
pkg/externalrepo/fake/repository_test.go	Updates expectations for fake UpdatePackageRevision behavior.
pkg/engine/pushpr.go	Removes legacy engine push implementation.
pkg/cache/dbcache/util.go	Adds push state helpers, keyed locking utilities, and push marker helpers.
pkg/cache/dbcache/util_test.go	Adds unit tests for new util helpers.
pkg/cache/dbcache/dbreposync.go	Reworks cached-only handling and adds sync-time push enqueueing for draft/proposed.
pkg/cache/dbcache/dbreposync_test.go	Updates/extends repo-sync unit tests for new cached-only + push behavior.
pkg/cache/dbcache/dbrepository.go	Removes git PR cache and adds locking + task handling changes for DB repo ops.
pkg/cache/dbcache/dbrepository_test.go	Updates repository tests to match new DB-cache push model.
pkg/cache/dbcache/dbpushtogit.go	Adds DB-cache push implementation for published and draft/proposed revisions.
pkg/cache/dbcache/dbpushtogit_test.go	Updates tests to use the new DB-cache push functions.
pkg/cache/dbcache/dbpackagerevisionsql.go	Adds last_pushed_db_updated column handling + marker update helper.
pkg/cache/dbcache/dbpackagerevision.go	Adds push markers to dbPackageRevision and routes publish through new push logic.
pkg/cache/dbcache/dbpackagerevision_test.go	Updates publish flow tests for new push implementation.
make/deploy.mk	Adds DB_PUSH_DRAFTS_TO_GIT variable and enables it for kind v1alpha2 target.
deployments/porch/9-controllers.yaml	Adds controller arg for --repositories.push-drafts-to-git.
deployments/porch/3-porch-postgres-bundle.yaml	Adds last_pushed_db_updated column to bundled schema.
api/sql/porch-db.sql	Adds last_pushed_db_updated column to base DB schema.
api/sql/porch-db-1.6.4-1.6.0.sql	Adds downgrade migration (drop last_pushed_db_updated).
api/sql/porch-db-1.6.0-1.6.4.sql	Adds upgrade migration (add last_pushed_db_updated).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[rendre-greyling]

Whilst this is true, the volume expected for these pushes is very low. Fix would be to put this in a worker pool and limit the number of parallel pushes introducing more cmd args.

I would prefer to hold off on this now as there is a common worker pool implementation in the works and would rather use that.

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
86.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud