feat: ORM SQLAlchemyDataLayer

[hayescode]

Refactor SQLAlchemyDataLayer to use ORM

Summary

Refactors sql_alchemy.py from raw SQL queries to SQLAlchemy ORM, improving maintainability and adding multi-dialect database support.

Benefits

• Multi-dialect support: Now works with PostgreSQL, SQLite, and MySQL/MariaDB out of the box
• Type safety: ORM models provide better IDE autocompletion and type checking
• Reduced code duplication: Removed ~200 lines of repetitive SQL string construction
• Automatic schema creation: New create_tables=True parameter auto-creates tables on first use
• Easier testing: Test fixtures reduced from 100+ lines of manual SQL to 4 lines

What's New

File	Description
chainlit/data/models.py	New ORM models with CrossDialectJSON type for PostgreSQL/SQLite/MySQL compatibility
chainlit/data/sql_alchemy.py	Refactored to use ORM operations instead of raw SQL

Migration

No breaking changes - the public API remains identical.

Optional: Users can now enable automatic table creation:

from chainlit.data.sql_alchemy import SQLAlchemyDataLayer

data_layer = SQLAlchemyDataLayer(
    conninfo="sqlite+aiosqlite:///data.db",
    create_tables=True,  # NEW: Auto-creates tables if they don't exist
)

---

Summary by cubic

Refactors SQLAlchemyDataLayer to use SQLAlchemy ORM, adding multi-dialect support and optional automatic table creation. Improves maintainability and type safety, keeps the public API unchanged, and simplifies tests.

• New Features

  • Works with PostgreSQL, SQLite, and MySQL/MariaDB.
  • Optional create_tables=True to auto-create tables.
  • CrossDialectJSON for JSONB on Postgres and JSON elsewhere.

• Refactors

  • Replaced raw SQL with ORM models and session-based operations; unified per-dialect upsert.
  • Removed SQL helpers and duplicated query code.
  • Simplified tests by enabling automatic schema creation.

^{Written for commit 469274c. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 3 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="backend/chainlit/data/sql_alchemy.py">

<violation number="1" location="backend/chainlit/data/sql_alchemy.py:615">
P2: `autoPlay` is read from a non-existent attribute on `Element`, so auto-play settings are always saved as NULL. Use the `element_dict` (or `element.auto_play`) instead.</violation>

<violation number="2" location="backend/chainlit/data/sql_alchemy.py:616">
P2: `playerConfig` is read from a non-existent attribute on `Element`, so video player configuration never persists. Use the `element_dict` (or `element.player_config`) instead.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[Sanady]

Hey @hayescode, is this pull request going to fix the issue that I reported? #2758

[dominpm]

If we are currently using the official ChainlitDataLayer with a postgresql and a blob container in Azure for storage_client i.e.:

@data_layer
def get_data_layer():
    return ChainlitDataLayer(
        database_url=DATABASE_URL,
        storage_client=AzureBlobStorageClient(
            container_name=DATALAYER_AZURE_BLOB_CONTAINER_NAME,
            storage_account=DATALAYER_AZURE_BLOB_STORAGE_ACCOUNT_NAME,
            storage_key=DATALAYER_AZURE_BLOB_STORAGE_ACCOUNT_KEY
        )
    )

should we migrate to this SQLAlchemy data layer?

And if so, is there a direct way?

PS: Is this datalayer still maintained https://github.com/Chainlit/chainlit-datalayer?

[hayescode]

That data layer is not maintained. The goal here is to consolidate the mess of data layers.