Skip to content

fix(autocomplete): skip server-wide schema indexing on servers with many databases - #353

Open
arturonline wants to merge 1 commit into
Maxteabag:mainfrom
arturonline:fix/autocomplete-many-databases
Open

arturonline wants to merge 1 commit into
Maxteabag:mainfrom
arturonline:fix/autocomplete-many-databases

Conversation

@arturonline

Copy link
Copy Markdown

Problem

Connecting to a SQL Server instance with many databases, without a default database set, makes sqlit unresponsive for minutes. It often has to be killed and restarted.

Reproduced against a real server with 236 user databases (about 280k tables, 209k views, 594k routines and 3M routine parameters in total).

Cause

When no database is selected and the provider supports cross-database queries, the autocomplete schema loader indexes every non-system database on the server (_load_schema_directly → get_databases → _on_databases_loaded(all_dbs)).

For each database it runs 3 metadata queries: tables, views, and routines with parameters. Each query is fast on its own (about 1s per database in total), but:

  1. Executor starvation. All 3 × N queries (about 700 here) go through session.executor, which is the single thread that owns the connection. Anything the user does (expanding a tree node, running a query) waits behind them for several minutes.
  2. UI thread load. Every table and view adds 3–4 entries to _table_metadata, and every routine is appended to the cache. That is millions of items processed in idle-scheduler batches on the main thread, plus several hundred worker threads spawned at once.

The result looks like a hang.

Fix

Add MAX_SERVER_WIDE_INDEX_DATABASES = 20. When the server-wide database list is longer than this, autocomplete indexing is skipped and a warning is shown:

Autocomplete not indexed: the server has N databases. Select a database in the explorer (Enter on it) to index it.

Selecting a database already calls _load_schema_cache() with that database as the only target, so indexing picks up there with no further changes. Servers with 20 or fewer databases, and connections with a database configured, behave exactly as before.

The guard is applied in both loading paths:

  • _on_databases_loaded (threaded / idle-scheduler path). A selected database always arrives here as a single-item list, so it is never affected.
  • _load_schema_cache_async, right after the database list is fetched.

Workaround for users on older versions

Set a database on the connection. Autocomplete then indexes only that database.

Tests

New tests/unit/test_autocomplete_many_databases.py:

  • more than the limit: nothing is indexed, the spinner stops, and one warning is emitted
  • at the limit: databases are indexed, no warning
  • a single selected database is always indexed

The new tests pass when run directly. I couldn't run the full suite with pytest in my local environment.

Possible follow-ups (not in this PR)

  • Make the limit configurable in settings.
  • Index the active database first and the rest lazily or on demand, instead of all or nothing.

🤖 Generated with Claude Code

…any databases

With no database selected, autocomplete indexed every user database on the
server. On a SQL Server instance with hundreds of databases that queued
hundreds of metadata queries on the single connection executor and processed
millions of entries on the UI thread, leaving the app unresponsive for
minutes.

Past MAX_SERVER_WIDE_INDEX_DATABASES (20) databases, indexing is skipped with
a warning; selecting a database indexes just that one, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant