Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error happened in Index building #542

Open
IANTHEREAL opened this issue Dec 25, 2024 · 0 comments
Open

Error happened in Index building #542

IANTHEREAL opened this issue Dec 25, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@IANTHEREAL
Copy link
Contributor

the error stack:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1144, in _commit_impl
    self.engine.dialect.do_commit(self.connection)
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 701, in do_commit
    dbapi_connection.commit()
  File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 483, in commit
    self._read_ok_packet()
  File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 453, in _read_ok_packet
    pkt = self._read_packet()
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 775, in _read_packet
    packet.raise_for_error()
  File "/usr/local/lib/python3.11/site-packages/pymysql/protocol.py", line 219, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.11/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1105, 'tikv aborts txn: Error(KeyTooLarge { size: 9342, limit: 8192 })')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/app/tasks/build_index.py", line 60, in build_index_for_document
    index_service.build_vector_index_for_document(index_session, db_document)
  File "/app/app/rag/build_index.py", line 72, in build_vector_index_for_document
    vector_index.insert(document, source_uri=db_document.source_uri)
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/indices/base.py", line 215, in insert
    self.insert_nodes(nodes, **insert_kwargs)
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/indices/vector_store/base.py", line 330, in insert_nodes
    self._insert(nodes, **insert_kwargs)
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/indices/vector_store/base.py", line 311, in _insert
    self._add_nodes_to_index(self._index_struct, nodes, **insert_kwargs)
  File "/usr/local/lib/python3.11/site-packages/llama_index/core/indices/vector_store/base.py", line 232, in _add_nodes_to_index
    new_ids = self._vector_store.add(nodes_batch, **insert_kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/app/rag/vector_store/tidb_vector_store.py", line 135, in add
    self._session.commit()
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2017, in commit
    trans.commit(_to_root=True)
  File "<string>", line 2, in commit
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
                ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1309, in commit
    trans.commit()
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2629, in commit
    self._do_commit()
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2734, in _do_commit
    self._connection_commit_impl()
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2705, in _connection_commit_impl
    self.connection._commit_impl()
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1146, in _commit_impl
    self._handle_dbapi_exception(e, None, None, None, None)
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2353, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1144, in _commit_impl
    self.engine.dialect.do_commit(self.connection)
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 701, in do_commit
    dbapi_connection.commit()
  File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 483, in commit
    self._read_ok_packet()
  File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 453, in _read_ok_packet
    pkt = self._read_packet()
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 775, in _read_packet
    packet.raise_for_error()
  File "/usr/local/lib/python3.11/site-packages/pymysql/protocol.py", line 219, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.11/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
    raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1105, 'tikv aborts txn: Error(KeyTooLarge { size: 9342, limit: 8192 })')
(Background on this error at: https://sqlalche.me/e/20/e3q8)
@IANTHEREAL IANTHEREAL added the bug Something isn't working label Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant