perf: Enhance objects using constant sql queries count to get rid of N+1 queries by Koc · Pull Request #2857 · nextcloud/tables

Koc · 2026-07-29T00:14:04Z

In our organization we operate a large amount of tables/views, so there are performance bottleneck here:

PERFORMANCE: Controller OCA\\Tables\\Controller\\Api1Controller::index executed 1883 queries in 20.1 seconds
PERFORMANCE: Controller OCA\\Tables\\Controller\\TableController::index executed 1887 queries in 16.6 seconds

Replaced TableService::enhanceTable() in a loop with a single call of the enhanceTables()
Replaced ViewService::enhanceView() in a loop with a single call of theenhanceViews()
Added batch helpers in mappers/services and UserHelper for row/column counts, share counts and owner display names.

🖼️ Screenshots

Here profiler results for GET /apps/tables/table endpoint for 18 tables and 9 views:

🏚️ Before	🏡 After
121 queries	29 queries

🏁 Checklist

⛑️ Tests (unit and/or integration) are included or not needed
🔙 Backport requests are created or not needed: /backport to stableX.X
📅 Milestone is set
🌸 PR title is meaningful (if it should be in the changelog: is it meaningful to users?)

🤖 AI (if applicable)

The content of this PR was partly or fully generated using AI

Koc · 2026-08-30T15:39:42Z

@enjeck I've fixed conflicts introduced by #2781 and code review comments, please, check one more time.

enjeck

a question:

blizzz

Some questions, and view ordering got lost it seems

blizzz · 2026-09-16T12:16:12Z

+	 * @param array<int, int|null> $tableRowsCounts
+	 */
+	private function enhanceViews(array $views, string $userId, array $tableRowsCounts = []): void {
+		if (empty($views)) {


this does not have the isFederated guard anymore. Intentional? It puts the responsibility to check about it to the caller, like in enhanceView() currently.

I've added behat test for that, please check

blizzz · 2026-09-16T12:21:22Z

+
+		try {
+			$excluded = !$this->circleHelper->isCirclesEnabled() ? [ShareReceiverType::CIRCLE] : [];
+			$shares = $this->mapper->findAllSharesForNodes('table', $tableIds, $userId, $excluded);


Does this still find all shares for a table manager, i.e. indirect shares?

I've added behat test for that, please check

Koc · 2026-10-09T21:23:52Z

@blizzz I've fixed code review comments, please check

…N+1 queries Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>

…N+1 queries (code review new portion) Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>

…N+1 queries (more tests) Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>

enjeck

I have some more comments:

enjeck · 2026-10-18T18:09:38Z

+		try {
+			$allViews = $this->mapper->findAllByTableIds($tableIds);


findForTables() dropped the canManageTable() check from the old findAll(). I think this causes a read-only sharee calling GET /api/2/tables/scheme/{id} gets 200 with all views and their filters ??

enjeck · 2026-10-18T18:14:32Z

 	public function getScheme(int $id, ?string $userId = null): TableScheme {
 		$table = $this->find($id, skipTableEnhancement: true);
 		$columns = $this->columnService->findAllByTable($id, null, $table);
-		$this->enhanceTable($table, $userId ?? '');


Should this be preCheckUserId($userId) instead?

enjeck · 2026-10-18T18:27:54Z

+			try {
+				$this->enhanceTables($allTables, $userId);
+			} catch (InternalError|PermissionError $e) {
+				$this->logger->error($e->getMessage(), ['exception' => $e]);


One exception here now skips enhancement for every table. Can each step handle its own errors like before?

enjeck · 2026-10-18T18:29:27Z

-			$this->logger->warning('Exception occurred: ' . $e->getMessage() . ' Returning 0.');
-			return 0;
+		$counts = array_fill_keys($tableIds, 0);
+		foreach (array_chunk($tableIds, 1000 - 1) as $tableIdsChunk) {


Can we build the query once outside the loop and use a DB_CHUNK_SIZE constant?

same in ther places

enjeck · 2026-10-18T18:30:28Z

+	 * @param int[] $tableIds
+	 * @return int[]
+	 */
+	public function findLinkSharesForTables(array $tableIds): array {


Suggested change

public function findLinkSharesForTables(array $tableIds): array {

public function findTableIdsWithLinkShares(array $tableIds): array {

enjeck · 2026-10-18T18:31:54Z

+	 * @return array<int, int>
+	 * @throws InternalError
+	 */
+	public function countSharesForTables(array $tableIds): array {


This duplicates countSharesForViews(). Could they merge into one countSharesForNodes($nodeType, $nodeIds)? Or is that too hard ?

Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 9b6bd68 to 4033598 Compare July 29, 2026 00:28

Koc marked this pull request as ready for review July 29, 2026 18:08

Koc requested review from blizzz and enjeck as code owners July 29, 2026 18:08

Koc changed the title ~~perf: Call enhanceTables for a multiple tables at once to get rid of N+1 queries~~ perf: Enhance objects for a multiple tables at once to get rid of N+1 queries Jul 29, 2026

Koc changed the title ~~perf: Enhance objects for a multiple tables at once to get rid of N+1 queries~~ perf: Enhance objects using constant sql queries count to get rid of N+1 queries Jul 29, 2026

Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 29cbb47 to 44e315b Compare July 29, 2026 21:57

enjeck reviewed Aug 7, 2026

View reviewed changes

Comment thread lib/Db/ShareMapper.php Outdated

Comment thread lib/Service/ViewService.php Outdated

Comment thread lib/Db/RowSleeveMapper.php Outdated

Comment thread lib/Controller/Api1Controller.php Outdated

Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 2686136 to e9b83fb Compare August 9, 2026 14:30

Koc requested a review from enjeck August 9, 2026 14:46

samin-z self-requested a review August 10, 2026 15:48

Koc force-pushed the feature/optimize-enhance-tables branch from e9b83fb to b286459 Compare August 24, 2026 00:51

enjeck reviewed Aug 25, 2026

View reviewed changes

Comment thread lib/Db/ShareMapper.php Outdated

Comment thread lib/Controller/Api1Controller.php

Comment thread lib/Service/ShareService.php

Comment thread lib/Controller/Api1Controller.php

Comment thread lib/Db/ColumnMapper.php

Comment thread lib/Db/ViewMapper.php Outdated

Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 306f894 to f5cb044 Compare August 27, 2026 23:58

nextcloud-command added the AI assisted label Aug 27, 2026

Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from a7383f5 to c9e19d1 Compare August 30, 2026 14:27

enjeck reviewed Sep 4, 2026

View reviewed changes

Comment thread lib/Db/RowSleeveMapper.php Outdated

enjeck force-pushed the feature/optimize-enhance-tables branch from 263e65c to ba01275 Compare September 4, 2026 04:44

Koc force-pushed the feature/optimize-enhance-tables branch from ba01275 to ecef56b Compare September 4, 2026 08:27

Koc requested a review from enjeck September 4, 2026 09:05

Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 7243f39 to e845c15 Compare September 12, 2026 15:42

blizzz requested changes Sep 16, 2026

View reviewed changes

Koc force-pushed the feature/optimize-enhance-tables branch from e845c15 to ecef56b Compare September 16, 2026 22:53

Koc force-pushed the feature/optimize-enhance-tables branch from ecef56b to abc0f82 Compare September 16, 2026 22:54

Koc force-pushed the feature/optimize-enhance-tables branch 4 times, most recently from ef7479b to e460edf Compare October 9, 2026 21:01

Koc requested a review from blizzz October 9, 2026 22:17

enjeck force-pushed the feature/optimize-enhance-tables branch from e460edf to b5cf342 Compare October 17, 2026 14:38

Koc added 3 commits October 18, 2026 19:26

perf: Enhance objects using constant sql queries count to get rid of …

409a51c

…N+1 queries Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>

perf: Enhance objects using constant sql queries count to get rid of …

eb1588e

…N+1 queries (code review new portion) Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>

perf: Enhance objects using constant sql queries count to get rid of …

70976c6

…N+1 queries (more tests) Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>

Koc force-pushed the feature/optimize-enhance-tables branch from b5cf342 to 70976c6 Compare October 18, 2026 17:26

enjeck reviewed Oct 18, 2026

View reviewed changes

		try {
		$allViews = $this->mapper->findAllByTableIds($tableIds);

	public function findLinkSharesForTables(array $tableIds): array {
	public function findTableIdsWithLinkShares(array $tableIds): array {

Conversation

Koc commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🖼️ Screenshots

🏁 Checklist

🤖 AI (if applicable)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Koc commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

enjeck left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

blizzz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Koc commented Oct 9, 2026

Uh oh!

enjeck left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Koc commented Jul 29, 2026 •

edited

Loading

Koc commented Aug 30, 2026 •

edited

Loading