Exception in thread "main" root cause "mapper_parsing_exception"

[maziyarpanahi]

Hi,

When I run the wikiparse with sample data I get this error:

java -jar -Xmx3g -Xms3g wikiparse-1.2.5-standalone.jar --es http://10.0.0.25:9200 wikisample.xml.bz2
Deleting index en-wikipedia
Creating index en-wikipedia
Exception in thread "main" clojure.lang.ExceptionInfo: clj-http: status 400 {:object {:orig-content-encoding nil, :trace-redirects ["http://10.0.0.25:9200/en-wikipedia"], :request-time 88, :status 400, :headers {"Content-Length" "459", "Content-Type" "application/json; charset=UTF-8"}, :body "{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"analyzer on completion field [suggest] must be set when search_analyzer is set"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [page]: analyzer on completion field [suggest] must be set when search_analyzer is set","caused_by":{"type":"mapper_parsing_exception","reason":"analyzer on completion field [suggest] must be set when search_analyzer is set"}},"status":400}"}, :environment {client #<client$wrap_output_coercion$fn__3762 clj_http.client$wrap_output_coercion$fn__3762@6bfdb014>, req {:headers {"accept" "application/json"}, :request-method :post, :url "http://10.0.0.25:9200/en-wikipedia", :body "{"settings":{"index":{"number_of_replicas":0,"gateway":{"local":{"sync":"60s"}},"translog":{"flush_threshold_size":"756mb","interval":"60s"},"number_of_shards":1,"refresh_interval":60}},"mappings":{"page":{"properties":{"redirects":{"fields":{"redirects_snow":{"type":"string","analyzer":"snowball"},"redirects_simple":{"type":"string","analyzer":"simple"},"redirects_exact":{"index":"not_analyzed","type":"string"}},"type":"string"},"format":{"index":"not_analyzed","type":"string"},"ns":{"index":"not_analyzed","type":"string"},"title":{"fields":{"title_snow":{"type":"string","analyzer":"snowball"},"title_exact":{"index":"not_analyzed","type":"string"},"title_simple":{"type":"string","analyzer":"simple"}},"type":"string"},"suggest":{"index_analyzer":"simple","type":"completion","search_analyzer":"simple"},"redirect":{"index":"not_analyzed","type":"string"},"timestamp":{"type":"date"},"body":{"fields":{"body_snow":{"type":"string","analyzer":"snowball"},"body_simple":{"type":"string","analyzer":"simple"}},"type":"string"}},"_all":{"_enabled":false}}}}"}, map__3646 {:orig-content-encoding nil, :trace-redirects ["http://10.0.0.25:9200/en-wikipedia"], :request-time 88, :status 400, :headers {"Content-Length" "459", "Content-Type" "application/json; charset=UTF-8"}, :body "{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"analyzer on completion field [suggest] must be set when search_analyzer is set"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [page]: analyzer on completion field [suggest] must be set when search_analyzer is set","caused_by":{"type":"mapper_parsing_exception","reason":"analyzer on completion field [suggest] must be set when search_analyzer is set"}},"status":400}"}, resp {:orig-content-encoding nil, :trace-redirects ["http://10.0.0.25:9200/en-wikipedia"], :request-time 88, :status 400, :headers {"Content-Length" "459", "Content-Type" "application/json; charset=UTF-8"}, :body "{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"analyzer on completion field [suggest] must be set when search_analyzer is set"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [page]: analyzer on completion field [suggest] must be set when search_analyzer is set","caused_by":{"type":"mapper_parsing_exception","reason":"analyzer on completion field [suggest] must be set when search_analyzer is set"}},"status":400}"}, status 400}}
at clj_http.client$wrap_exceptions$fn__3645.invoke(client.clj:171)
at clj_http.client$wrap_accept$fn__3802.invoke(client.clj:521)
at clj_http.client$wrap_accept_encoding$fn__3808.invoke(client.clj:536)
at clj_http.client$wrap_content_type$fn__3797.invoke(client.clj:512)
at clj_http.client$wrap_form_params$fn__3878.invoke(client.clj:683)
at clj_http.client$wrap_nested_params$fn__3895.invoke(client.clj:707)
at clj_http.client$wrap_method$fn__3848.invoke(client.clj:624)
at clj_http.cookies$wrap_cookies$fn__704.invoke(cookies.clj:122)
at clj_http.links$wrap_links$fn__2197.invoke(links.clj:50)
at clj_http.client$wrap_unknown_host$fn__3903.invoke(client.clj:726)
at clj_http.client$post.doInvoke(client.clj:841)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at clojurewerkz.elastisch.rest$post.invoke(rest.clj:48)
at clojurewerkz.elastisch.rest.index$create.doInvoke(index.clj:55)
at clojure.lang.RestFn.invoke(RestFn.java:521)
at wikiparse.core$ensure_index.invoke(core.clj:190)
at wikiparse.core$_main.doInvoke(core.clj:268)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at wikiparse.core.main(Unknown Source)

There is no info on the mapping on the readme. Is mapping being created automatically on the creation of the index or should be set before manually?

Update: I just looked at the source and saw the mapping is being set at the creation time automatically. So what could cause this error?

Elasticsearch v2.1

Thanks a lot,
Maziyar

[pippobaudos]

Hi Maziyar, I am getting the same error on the versions 2.0.0 and 2.1.1
Instead, on elasticsearch-1.7.1, everything is working fine! : )

[krisleech]

Same here for ES 2.1.1

[maziyarpanahi]

Well apparently Elasticsearch 2.0 above changed the way you should create field with type completion. The order of those few lines are the ones that is causing the error. You can create the index manually with the same config and you will see the same error:

Failed to parse mapping [page]: analyzer on completion field [suggest] must be set when search_analyzer is set

I think this:

:suggest {
               :type :completion
               :index_analyzer :simple
               :search_analyzer :simple}

should change to this:

"suggest": {
          "search_analyzer": "simple",
          "analyzer": "simple",
          "type": "completion"
        }

The type should appear after setting the analyzer. I know this shouldn't be the cause of the crash during the creation of an index and Elasticsearch should be smarter to know analyzer is already set in the same clause, but for know this will be the issue in ES version 2.0 and above.

Please let me know if I am wrong.