index not found error when following setup readme.

[annashch-insitro]

Hello, I'm setting up snovault on Mac OSX (Catalina), and following the instructions here: https://github.com/ENCODE-DCC/snovault/blob/dev/README.md

When running

bin/dev-servers development.ini --app-name app --clear --init --load

I get the following error:

[2020-11-01T21:13:26,934][INFO ][o.e.n.Node               ] [DN52eg12.SUNet] started
detected start, broke
returning process
Started: http://localhost:8000
Redis Config created: /tmp/snovault/redisdata/redis.conf
Starting redis server: redis-server /tmp/snovault/redisdata/redis.conf
Connect with 'redis-cli -p 6378 -n 1'
CREATE MAPPING RUNNING
Traceback (most recent call last):
  File "bin/dev-servers", line 76, in <module>
    sys.exit(snovault.dev_servers.main())
  File "/Users/annashch/snovault/src/snovault/dev_servers.py", line 115, in main
    create_mapping.run(app)
  File "/Users/annashch/snovault/src/snovault/elasticsearch/create_mapping.py", line 517, in run
    create_snovault_index_alias(es, indices)
  File "/Users/annashch/snovault/src/snovault/elasticsearch/create_mapping.py", line 484, in create_snovault_index_alias
    es.indices.put_alias(index=','.join(indices), name=RESOURCES_INDEX, request_timeout=300)
  File "/Users/annashch/snovault/eggs/elasticsearch-5.4.0-py3.7.egg/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/Users/annashch/snovault/eggs/elasticsearch-5.4.0-py3.7.egg/elasticsearch/client/indices.py", line 353, in put_alias
    '_alias', name), params=params, body=body)
  File "/Users/annashch/snovault/eggs/elasticsearch-5.4.0-py3.7.egg/elasticsearch/transport.py", line 312, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/Users/annashch/snovault/eggs/elasticsearch-5.4.0-py3.7.egg/elasticsearch/connection/http_urllib3.py", line 128, in perform_request
    self._raise_error(response.status, raw_data)
  File "/Users/annashch/snovault/eggs/elasticsearch-5.4.0-py3.7.egg/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.NotFoundError: TransportError(404, 'index_not_found_exception', 'no such index [award]')
2020-11-01 21:13:27.921 PST [25296] LOG:  received smart shutdown request

Do you have any suggestions?
Thank you!

[keenangraham]

It looks like the award index in ElasticSearch is never created for some reason, and it throws an error when the code later tries to alias that index. I would suspect maybe a faulty ES installation.

As a first step could you pull the latest dev and rebuild the application, then try running dev-servers again?

$ git checkout dev
$ git pull
$ make clean && buildout bootstrap && bin/buildout
$ bin/dev-servers development.ini --app-name app --clear --init --load

You could also try running some tests that use the ES fixture and see what happens, e.g.:

$ bin/test -s -k 'test_searchv2.py'

If that doesn't help I would say try reinstalling ElasticSearch, make sure you are using correct JVM version etc.

[annashch-insitro]

just following up on this -- the trick was to use a pre-built elasticsearch 5.6 binary for linux:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.0.tar.gz 

btw, I have a docker file that replicates the steps I used to get snovault running on ubuntu 18.04, if of interest I can submit a pull request? (I noticed the readme focuses on osx)

[keenangraham]

Yeah, that would be great. Can't promise we would merge it right away but I know we are looking for ways to make deployment easier so I'm sure @ottojolanki and @paul-sud would be interested to see Docker approach. Thanks.

[annashch-insitro]

pull request (#308) submitted.