Improve performance of large updates

[thegedge]

We've been seeing some slower updates recently. These are updates for some pretty substantial application file systems (approximately 1.8G), but we still would expect Dateilager (DL) to be able to update a filesystem of that size in under 30 seconds (if we could get that closer to 10 seconds, or even less, that would be amazing!). It looks kind of like this:

So what can we do in DL? From the above, this is the slow codepath that made up the slow span:

dateilager/pkg/api/fs.go

Lines 627 to 647 in 140e3dc

	err = telemetry.Trace(ctx, "update-packed-objects", func(ctx context.Context, span trace.Span) error {
	for parent, objects := range packedBuffer {
	logger.Debug(ctx, "FS.Update[PackedObject]",
	key.Project.Field(project),
	key.Version.Field(nextVersion),
	key.ObjectsParent.Field(parent),
	key.ObjectsCount.Field(len(objects)),
	)
	contentChanged, err := db.UpdatePackedObjects(ctx, tx, f.DbConn, project, nextVersion, parent, objects)
	if err != nil {
	return status.Errorf(codes.Internal, "FS update packed objects for %v: %v", parent, err)
	}
	if contentChanged {
	shouldUpdateVersion = true
	}
	}
	return nil
	})

The first line that jumps out to me is:

for parent, objects := range packedBuffer {

Are there a lot of objects? I think so, but we should investigate that. If there are lots, is there any way to batch the DB queries we make, so there's just one big SELECT and a bulk INSERT, instead of one per object.

Given the above trace, I think that's what's making the Update RPC so slow.

If there aren't lots of objects, we'll need to investigate this a bit more to understand why there isn't a lot.

[thegedge]

In terms of reproduction, I'd recommend setting up a JS project with a lot of large dependencies. Perhaps try something like this:

  "dependencies": {
    "@effect-rx/rx": "^0.34.2",
    "@effect-rx/rx-react": "^0.31.2",
    "@effect/opentelemetry": "^0.39.12",
    "@effect/schema": "^0.75.5",
    "@effect/vitest": "^0.13.12",
    "@gadgetinc/api-client-core": "^0.15.40",
    "@gadgetinc/ggt": "^1.4.1",
    "@gadgetinc/react": "^0.18.7",
    "@gadgetinc/react-shopify-app-bridge": "^0.16.7",
    "@gadgetinc/shopify-extensions": "^0.3.5",
    "@graphql-codegen/cli": "^5.0.2",
    "@mollie/api-client": "^3.7.0",
    "@opentelemetry/auto-instrumentations-node": "^0.50.0",
    "@opentelemetry/exporter-trace-otlp-http": "^0.53.0",
    "@opentelemetry/sdk-trace-node": "^1.26.0",
    "@opentelemetry/sdk-trace-web": "^1.26.0",
    "@quasarwork/gear": "latest",
    "@react-email/components": "0.0.24",
    "@sentry/node": "^8.9.2",
    "@sentry/opentelemetry": "^8.28.0",
    "@sentry/profiling-node": "^8.28.0",
    "@shopify/admin-api-client": "^1.0.1",
    "@shopify/admin-graphql-api-utilities": "^2.2.0",
    "@shopify/api-codegen-preset": "^1.0.1",
    "@shopify/app": "^3.58.2",
    "@shopify/app-bridge": "^3.7.10",
    "@shopify/app-bridge-react": "^4.1.3",
    "@shopify/cli": "^3.63.1",
    "@shopify/polaris": "^13.6.0",
    "@shopify/polaris-icons": "^9.2.0",
    "@shopify/ui-extensions": "^2024.4.1",
    "@shopify/ui-extensions-react": "^2024.4.1",
    "@skorotkiewicz/snowflake-id": "^1.0.1",
    "@vanilla-extract/css": "^1.16.0",
    "date-fns": "^3.6.0",
    "effect": "^3.10.12",
    "fastify": "^4.24.2",
    "fastify-raw-body": "4.3",
    "gocardless-nodejs": "^3.28.0",
    "i18next": "^23.16.5",
    "jiti": "^2.4.0",
    "pino": "^9.5.0",
    "query-string": "^9.1.1",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-email": "^3.0.1",
    "react-error-boundary": "^4.0.13",
    "react-i18next": "^15.1.1",
    "react-reconciler": "^0.29.2",
    "react-router-dom": "^6.21.1",
    "resend": "^4.0.0",
    "shopify": "^3.66.1",
    "shopify-api-node": "^3.13.1",
    "svix": "^1.40.0",
    "transliteration": "^2.3.5",
    "valibot": "^0.40.0"
  },
  "devDependencies": {
    "@shopify/app-bridge-types": "^0.0.14",
    "@types/node": "^20.12.8",
    "@types/react": "^18.0.25",
    "@types/react-dom": "^18.0.9",
    "@typescript-eslint/parser": "^8.14.0",
    "@typescript-eslint/utils": "^8.14.0",
    "@vanilla-extract/vite-plugin": "^4.0.17",
    "@vitejs/plugin-react-swc": "3.7.0",
    "@vitest/ui": "^2.0.5",
    "cross-env": "^7.0.3",
    "dotenv": "^16.3.1",
    "eslint": "^9.14.0",
    "eslint-plugin-prettier": "^5.2.1",
    "eslint-plugin-react": "^7.37.2",
    "eslint-plugin-react-hooks": "^5.0.0",
    "eslint-plugin-react-refresh": "^0.4.14",
    "globals": "^15.12.0",
    "prettier": "^3.3.2",
    "ts-node": "^10.9.2",
    "typescript": "^5.6.3",
    "typescript-eslint": "^8.14.0",
    "vite": "^6.1.0",
    "vitest": "^3.0.5"
  }