Skip to content

[bug] IntegrityError When Adding Texts to Vector Store with Metadata #213

Open
@Ocyss

Description

@Ocyss

I have written the following code based on the example documentation. However, I encountered an error when attempting to add texts to the vector store with metadata.

Code

Here is the relevant code snippet:

pg_engine_entity = PGEngine.from_engine(engine)

await pg_engine_entity.ainit_vectorstore_table(
    table_name=TABLE_NAME_ENTITY,
    vector_size=1024,
    overwrite_existing=True,
    metadata_columns=[
        Column("entity_id", "INTEGER", nullable=False)
    ],
)

vectorstore_entity = await PGVectorStore.create(
    engine=pg_engine_entity,
    table_name=TABLE_NAME_ENTITY,
    embedding_service=openai_embeddings,
)

vid = (
    await vectorstore.aadd_texts(
        [memory_flow.content],
        [
            {
                "entity_id": entity_id
            }
        ],
        [uuid.uuid4()],
    )
)[0]

Error Message

When executing the code, I received the following error:

IntegrityError: (sqlalchemy.dialects.postgresql.asyncpg.IntegrityError) <class 'asyncpg.exceptions.NotNullViolationError'>: null value in column "entity_id" of relation "VectorEntity" violates not-null constraint
DETAIL: Failing row contains (63c145e7-8663-471e-845e-d96714f651e2, 一条测试。, [-0.030918628,0.017731816,-0.03639662,-0.00945296,-0.03324449,-0..., null, null, {"entity_id": 1006}).

Expected Behavior

The text and metadata should be successfully added to the vector store without violating the not-null constraint on the entity_id column.

Actual Behavior

The operation fails with an IntegrityError due to a null value in the entity_id column, despite passing a value in the metadata.

Environment

  • Library Version:
    • SQLModel: 0.0.24
    • asyncpg: 0.30.0
    • FastAPI: 0.115.12
    • LangChain: 0.3.25
    • Pydantic: 2.11.4
  • Database: PostgreSQL
  • Python Version: 3.13.2
  • Other Relevant Info: Package manager: uv

Additional Notes

  • It seems that the metadata value for entity_id is not being correctly mapped to the column in the database.
  • I suspect there might be an issue with how the metadata is processed or inserted into the table.

Request for Help

Could you please help identify why the entity_id value from the metadata is not being inserted into the database column? Any guidance or fixes would be greatly appreciated!

Additional Question on SQL Construction

I also noticed that the add_embeddings method uses f-strings to manually concatenate SQL statements. Could this potentially lead to security issues, and is there a reason for not using SQLAlchemy's table objects or parameterized queries?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions