Cache problem with local instance

[fescudie]

Hello,

From an HGVSg I try to get the normalized HGVSg, HGVSc and HGVSp via mutalyzer/json/runMutalyzerLight.
On the main instance of mutalyzer no problem. On a local instance I have several problems.
When using the web interface or the JSON API I have two problems:

• My variants are located on exonic, intronic or intergenic sequences. In this configuration, mutalyser needs sequences for the normalization. So, it automatically downloads the NC from the EBI into its cache. However, after downloading the NC sequence, it appears to download all transcripts and it fails. This error is probably due to the number of requests. I tried with an EBI API key and the result is the same. I only succeeded for a small chromosome.

127.0.0.1 - - [15/Oct/2019 16:17:36] "GET /name-checker?description=NC_000005.10%3Ag.83538811T%3EC HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 2463, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 2449, in wsgi_app
    response = self.handle_exception(e)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/mutalyzer-2.0.31-py2.7.egg/mutalyzer/website/views.py", line 248, in name_checker
    variantchecker.check_variant(description, output)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/mutalyzer-2.0.31-py2.7.egg/mutalyzer/variantchecker.py", line 1798, in check_variant
    retrieved_record = retriever.loadrecord(record_id)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/mutalyzer-2.0.31-py2.7.egg/mutalyzer/Retriever.py", line 661, in loadrecord
    filename = self.fetch(accession)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/mutalyzer-2.0.31-py2.7.egg/mutalyzer/Retriever.py", line 297, in fetch
    retmode='text')
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/Bio/Entrez/__init__.py", line 195, in efetch
    return _open(cgi, variables, post=post)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/lib/python2.7/site-packages/Bio/Entrez/__init__.py", line 566, in _open
    and exception.status // 100 == 4:
AttributeError: 'HTTPError' object has no attribute 'status'
127.0.0.1 - - [15/Oct/2019 16:17:36] "GET /name-checker?__debugger__=yes&cmd=resource&f=debugger.js HTTP/1.1" 200 -
127.0.0.1 - - [15/Oct/2019 16:17:36] "GET /name-checker?__debugger__=yes&cmd=resource&f=style.css HTTP/1.1" 200 -
127.0.0.1 - - [15/Oct/2019 16:17:36] "GET /name-checker?__debugger__=yes&cmd=resource&f=jquery.js HTTP/1.1" 200 -
127.0.0.1 - - [15/Oct/2019 16:17:37] "GET /name-checker?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 200 -
127.0.0.1 - - [15/Oct/2019 16:17:37] "GET /name-checker?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 200 -

• To get around this problem, I tried to synchronize the cache with the main instance of mutalyzer: mutalyzer-admin sync-cache 'https://mutalyzer.nl/services/?wsdl' 'https://mutalyzer.nl/Reference/{file}'. But this command fails because it inserts data without a "source" when this field cannot be null in the database.

(mutalyzer_env) user@computer:~/Documents/mutalyzer$ mutalyzer-admin sync-cache 'https://mutalyzer.nl/services/?wsdl' 'https://mutalyzer.nl/Reference/{file}'
Traceback (most recent call last):
  File "/home/user/Documents/mutalyzer/mutalyzer_env/bin/mutalyzer-admin", line 11, in <module>
    load_entry_point('mutalyzer==2.0.31', 'console_scripts', 'mutalyzer-admin')()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/mutalyzer-2.0.31-py2.7.egg/mutalyzer/entrypoints/admin.py", line 441, in main
    args.func(**{k: v for k, v in vars(args).items()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/mutalyzer-2.0.31-py2.7.egg/mutalyzer/entrypoints/admin.py", line 172, in sync_cache
    history)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/mutalyzer-2.0.31-py2.7.egg/mutalyzer/sync.py", line 184, in sync_with_remote
    session.commit()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1027, in commit
    self.transaction.commit()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 494, in commit
    self._prepare_impl()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 473, in _prepare_impl
    self.session.flush()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2459, in flush
    self._flush(objects)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2597, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2557, in _flush
    flush_context.execute()
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
    rec.execute(self)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
    uow,
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
    insert,
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1138, in _emit_insert_statements
    statement, params
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
    distilled_params,
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/home/user/Documents/mutalyzer/mutalyzer_env/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "source" violates not-null constraint
DETAIL:  Failing row contains (24, NM_001286191.2, a9ca5136d888c39876dd0dd95c60dac1, null, null, 2019-10-15 16:13:02.206861).

[SQL: INSERT INTO "references" (accession, checksum, source, source_data, added) VALUES (%(accession)s, %(checksum)s, %(source)s, %(source_data)s, %(added)s) RETURNING "references".id]
[parameters: {'source': None, 'source_data': None, 'added': datetime.datetime(2019, 10, 15, 16, 13, 2, 206861), 'accession': NM_001286191.2, 'checksum': a9ca5136d888c39876dd0dd95c60dac1}]
(Background on this error at: http://sqlalche.me/e/gkpj)

For the installation I followed the instructions on https://mutalyzer.readthedocs.io. I installed version v2.0.31 and the data for the GRCh38 assembly.

How can I load all the necessary information to normalize my data on my local instance? Am I wrong in the way I do it?

Thanks in advance