SearchResultHitTextMatchInfo#numTokensDropped should be a long

[Simon-Wittker]

org.typesense.model.SearchResultHitTextMatchInfo#numTokensDropped is an Integer but should be a Long.

Given the following response, Jacksons fails to deserialize the JSON since the long value does not fit into an integer.

{
  "facet_counts": [],
  "found": 1,
  "hits": [
    {
      "document": {
        "category_de": "Fragebaum zu psychischen Erkrankungen",
        "id": "234",
        "name_de": "Psychose",
        "synonyms_de": [
          "Psychose",
          "Schizophrenie",
          "manisch-depressive Erkrankung",
          "Bipolare Störung"
        ]
      },
      "highlight": {
        "synonyms_de": [
          {
            "matched_tokens": [],
            "snippet": "Psychose"
          },
          {
            "matched_tokens": [],
            "snippet": "Schizophrenie"
          },
          {
            "matched_tokens": [
              "manisch",
              "depressive"
            ],
            "snippet": "<mark>manisch</mark>-<mark>depressive</mark> Erkrankung"
          },
          {
            "matched_tokens": [],
            "snippet": "Bipolare Störung"
          }
        ]
      },
      "highlights": [
        {
          "field": "synonyms_de",
          "indices": [
            2
          ],
          "matched_tokens": [
            [
              "manisch",
              "depressive"
            ]
          ],
          "snippets": [
            "<mark>manisch</mark>-<mark>depressive</mark> Erkrankung"
          ]
        }
      ],
      "text_match": 1157451471441100921,
      "text_match_info": {
        "best_field_score": "2211897868288",
        "best_field_weight": 15,
        "fields_matched": 1,
        "num_tokens_dropped": 18446744073709551615,
        "score": "1157451471441100921",
        "tokens_matched": 2,
        "typo_prefix_score": 0
      }
    }
  ],
  "out_of": 253,
  "page": 1,
  "request_params": {
    "collection_name": "diagnosis",
    "first_q": "manischdepressive",
    "per_page": 10,
    "q": "manischdepressive"
  },
  "search_cutoff": false,
  "search_time_ms": 0
}

The following exception is thrown

com.fasterxml.jackson.databind.JsonMappingException: Numeric value (18446744073709551615) out of range of int (-2147483648 - 2147483647)
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 854] (through reference chain: org.typesense.model.SearchResult["hits"]->java.util.ArrayList[0]->org.typesense.model.SearchResultHit["text_match_info"]->org.typesense.model.SearchResultHitTextMatchInfo["num_tokens_dropped"])
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:401)
	at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:360)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1964)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:312)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:246)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4931)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3868)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3836)
	at org.typesense.api.ApiCall.handleResponse(ApiCall.java:265)
	at org.typesense.api.ApiCall.makeRequest(ApiCall.java:189)
	at org.typesense.api.ApiCall.get(ApiCall.java:131)
	at org.typesense.api.Documents.search(Documents.java:52)
	at org.example.APIClient.searchWithNumTypos(APIClient.java:169)
	at org.example.App.main(App.java:22)
Caused by: com.fasterxml.jackson.core.exc.InputCoercionException: Numeric value (18446744073709551615) out of range of int (-2147483648 - 2147483647)
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 854]
	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportInputCoercion(ParserMinimalBase.java:618)
null
	at com.fasterxml.jackson.core.base.ParserMinimalBase.reportOverflowInt(ParserMinimalBase.java:680)
	at com.fasterxml.jackson.core.base.ParserMinimalBase.reportOverflowInt(ParserMinimalBase.java:675)
	at com.fasterxml.jackson.core.base.ParserBase._reportTooLongIntegral(ParserBase.java:1040)
	at com.fasterxml.jackson.core.base.ParserBase._parseSlowInt(ParserBase.java:1023)
	at com.fasterxml.jackson.core.base.ParserBase._parseNumericValue(ParserBase.java:950)
	at com.fasterxml.jackson.core.base.ParserBase._parseIntValue(ParserBase.java:979)
	at com.fasterxml.jackson.core.base.ParserBase.getIntValue(ParserBase.java:787)
	at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:526)
	at com.fasterxml.jackson.databind.deser.std.NumberDeserializers$IntegerDeserializer.deserialize(NumberDeserializers.java:506)
	at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	... 20 more

Caused by: com.fasterxml.jackson.core.exc.InputCoercionException: Numeric value (18446744073709551615) out of range of int (-2147483648 - 2147483647)

[saw303]

It might need even a java.math.BigInteger in order to represent the actual value.

[saw303]

@Simon-Wittker just noticed that this is a duplicate of #84 and I guess you can close your issue favour of #84

[stevemcleod]

I think the underlying issue is that num_tokens_dropped has an incorrect value. I've reported this underlying issue to the main core Typesense GitHub project: typesense/typesense#2319

[kishorenc]

We've just published a new RC build 29.0.rc20 that contains the fix. I will be closing this issue since it's not related to the Java client code.