[Bug](lance) Vector search on a Float16/Float64 column with an IVF index aborts the BE

[FANNG1]

Search before asking

• I had searched in the issues and found no similar issues.

Version

branch-4.1 and master (both pin lance-c v0.1.2 in thirdparty/vars.sh, which builds against lance-rs 4.0.1).

What's Wrong?

vector_search() over a FixedSizeList<Float16> or FixedSizeList<Float64> column that has an IVF vector index panics inside the Lance Rust code, and because the FFI boundary has no catch_unwind, the panic aborts the whole BE process.

thread 'lance-cpu' panicked at arrow-array-57.3.0/src/cast.rs:840:33:
primitive array
*** SIGABRT ... received by PID <be pid>

The client only sees the connection die:

ERROR 1105 (HY000): RpcException, msg: send fragments failed.
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception

cast.rs:840 is the as_primitive downcast helper, so the index search path appears to assume Float32 vectors.

Scope, verified by isolating each case with a freshly restarted BE and counting new panics in be.out:

Vector column type	flat search (use_index=false)	indexed search
Float32	ok	ok
Float16	ok	BE aborts
Float64	ok	BE aborts
UInt8 (hamming index)	ok	ok

So the trigger is specifically indexed search on a non-Float32 float column. Flat search over the same columns is fine, which is why the existing all_types.lance fixture — which has five vector columns but no index at all — never surfaced it.

What You Expected?

A user query must never abort the backend. Either the search works, or it returns an error.

How to Reproduce?

The repo has no fixture combining a non-Float32 vector column with an index, so the table has to be created first (pylance, against the MinIO started by docker/thirdparties/run-thirdparties-docker.sh -c iceberg):

import io, lance, lance_namespace, pyarrow as pa, pyarrow.ipc as ipc
from lance_namespace_urllib3_client.models import CreateTableRequest

ns = lance_namespace.connect("dir", {
    "root": "s3://warehouse/lance",
    "storage.endpoint": "http://127.0.0.1:19001",
    "storage.access_key_id": "admin", "storage.secret_access_key": "password",
    "storage.region": "us-east-1", "storage.aws_allow_http": "true"})
so = {"endpoint": "http://127.0.0.1:19001", "access_key_id": "admin",
      "secret_access_key": "password", "region": "us-east-1", "allow_http": "true"}

DIM, ROWS = 16, 1024
emb = pa.FixedSizeListArray.from_arrays(
    pa.array([float(r + j) for r in range(ROWS) for j in range(DIM)], type=pa.float16()), DIM)
t = pa.table({"row_id": pa.array(range(1, ROWS + 1), type=pa.int64()), "embedding": emb})
buf = io.BytesIO()
with ipc.new_stream(buf, t.schema) as w:
    w.write_table(t)
loc = ns.create_table(CreateTableRequest(id=["doris", "f16_repro"]), buf.getvalue()).location
lance.dataset(loc, storage_options=so).create_index(
    "embedding", "IVF_FLAT", name="idx", metric="L2", num_partitions=4)

Then, from Doris:

CREATE CATALOG lance_repro PROPERTIES (
    "type" = "lance", "lance.catalog.type" = "filesystem",
    "warehouse" = "s3://warehouse/lance",
    "s3.endpoint" = "http://127.0.0.1:19001",
    "s3.access_key" = "admin", "s3.secret_key" = "password",
    "s3.region" = "us-east-1", "use_path_style" = "true");

-- aborts the BE
SELECT row_id FROM vector_search(
    "table" = "lance_repro.doris.f16_repro",
    "column" = "embedding",
    "query_vector" = "[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]",
    "top_k" = "3", "nprobes" = "4");

-- same table, same query, no index: works
SELECT row_id FROM vector_search(
    "table" = "lance_repro.doris.f16_repro",
    "column" = "embedding",
    "query_vector" = "[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]",
    "top_k" = "3", "use_index" = "false");

Swap pa.float16() for pa.float64() to reproduce the Float64 case.

Note when reading be.out: it is appended across restarts, so compare the count of panicked at lines before and after the query rather than just grepping for the message.

Analysis

Not a Doris misuse of the C ABI. Running the same query against the same tables through pylance 7.0.0 works for Float16, Float64 and Float32 alike, and it accepts a query vector of any float width against a Float16 column — so passing the column's native element type, which is what lance_reader.cpp and lance_scanner_nearest do, is correct usage. The fault is in the pinned lance-rs 4.0.1.

Rebuilding with lance-c v0.1.6 (which moves the lance dependency from crates.io 4.0.1 to a newer git rev) makes both cases pass, verified locally:

	lance-c 0.1.2	lance-c 0.1.6
Float16 + IVF_FLAT	BE aborts	returns 1, 2, 3
Float64 + IVF_FLAT	BE aborts	returns 1, 2, 3

#66698 is already upgrading lance-c to v0.1.6 on master. One extra note for whoever lands it: on macOS the newer dependency tree needs -framework IOKit added to the BE link libraries, otherwise doris_be fails to link with undefined IORegistryEntry* / IOService* symbols.

Two things worth treating separately from the version bump:

1. The FFI boundary has no panic guard. Upgrading fixes these two cases, but any panic! anywhere in lance-rs still takes the BE down. A catch_unwind around the lance-c calls, translating a panic into a Status, would contain the whole class. Two other reachable panics found the same way: vector_search on an Int8 column with NULLs (fixed_size_list_array.rs:142, still reproducible on v0.1.6), and building a hamming index on a float column (kmeans.rs:382).
2. Test coverage has a hole shaped exactly like this bug. Every indexed fixture is Float32, and the only multi-type fixture (all_types.lance) has no index, so "non-Float32 column with an index" was never exercised. [feature](lance) Verified IVF_FLAT vector index coverage and lance-c 0.1.6 upgrade #66512 adds fixtures for those cells plus a suite that queries them.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[Doris-Breakwater]

Breakwater-GitHub-Analysis-Slot: slot_a596ce74251b

Initial assessment

Confirmed, high-impact dependency bug. The report is sufficiently detailed and internally consistent to act on: indexed IVF search over FixedSizeList<Float16> / FixedSizeList<Float64> reaches a panic in the pinned Lance Rust implementation and terminates the BE. The flat-search and Float32 controls isolate the failing cell well, and the reported lance-c v0.1.2 -> v0.1.6 A/B result identifies a practical fix. This should not be treated as a Doris C-ABI misuse or worked around by forcing all query vectors to Float32.

The issue currently has no labels. It should remain open until the fix and non-Float32 indexed-search regression coverage have landed on both master and branch-4.1.

Evidence verified from code and current PR state

• Current upstream/master and upstream/branch-4.1 both still pin lance-c v0.1.2 in thirdparty/vars.sh.
• On branch-4.1, LanceTableReader::_configure_vector_search() decodes the query according to its declared element type and calls lance_scanner_nearest() with LANCE_DTYPE_FLOAT16, LANCE_DTYPE_FLOAT32, LANCE_DTYPE_FLOAT64, LANCE_DTYPE_UINT8, or LANCE_DTYPE_INT8. Every normal nonzero lance-c return is converted to a Doris Status. This corroborates the report that the native-width query is intentional and that an ordinary Lance error would be handled without killing the BE.
• In both lance-c v0.1.2 and v0.1.6, the ffi_try! helper only translates Result::Err; it does not catch Rust panics. More importantly, lance-c's release profile explicitly uses panic = "abort". Therefore v0.1.6 fixes the reported Float16/Float64 cases through its newer Lance dependency, but it does not contain the broader panic class.
• PR [chore](lance) update the thridparty about lance-c version  #66698 is open against master and currently changes the pin to v0.1.6, but its pushed diff does not add the required macOS IOKit framework. The v0.1.6 dependency lock includes sysinfo -> objc2-io-kit, which supports the reported IORegistryEntry* / IOService* linker failure and the need to validate the final BE link on both macOS architectures.
• PR [feature](lance) Verified IVF_FLAT vector index coverage and lance-c 0.1.6 upgrade #66512 is a draft against branch-4.1. As of head 7380b3f2ff92996f13f36f1213966eb02e84642a, its description documents the v0.1.6 and Float16/Float64 work, but the pushed tree still pins v0.1.2 and the committed generator/tests still create only Float32 indexed tables. The pushed contents need to be aligned with the description before this issue can rely on that PR for branch-4.1 coverage.

Missing information / remaining uncertainty

No additional information is required to accept and fix the reported crash. For an exact upstream root-cause attribution within lance-rs 4.0.1, however, please preserve one full Rust backtrace (RUST_BACKTRACE=full) and the exact Doris/lance-c commit SHAs used for the failing and passing runs. The current arrow-array downcast location proves a type-assumption panic but is not enough by itself to identify the precise Lance caller without guessing.

The separately reported Int8-with-NULL and invalid hamming-index panics are credible indications of the same containment weakness, but they have different trigger paths and should not be assumed to share this Float16/Float64 root cause.

Recommended next steps

1. Complete [chore](lance) update the thridparty about lance-c version  #66698 for master: add the macOS IOKit link dependency, record Linux plus macOS x86_64/arm64 build results, and run the Float16/Float64 IVF_FLAT queries with both indexed and use_index=false controls.
2. Update [feature](lance) Verified IVF_FLAT vector index coverage and lance-c 0.1.6 upgrade #66512 (or use a separate focused PR) for branch-4.1 so the pushed change actually contains the v0.1.6 pin, IOKit handling, indexed Float16/Float64 fixtures, and regression assertions. The test must prove that an index was used, not merely that correct nearest rows were returned after a silent flat fallback.
3. Backport/land the dependency fix and coverage on both affected branches, then close this issue once the four primary cells pass: Float16 indexed/flat and Float64 indexed/flat. Keep Float32 indexed and UInt8/hamming as controls.
4. Track FFI panic containment separately in lance-c. A viable design must first use an unwind-capable release profile, then wrap exported ABI entry points (ideally through a common helper) with catch_unwind and translate panic payloads to LanceErrorCode::Internal / lance_last_error_message(). catch_unwind with the current panic = "abort" profile cannot work. Add an end-to-end test demonstrating that an induced Rust panic returns an error and leaves the hosting process alive, including panics arising during scanner execution.
5. File the Int8-with-NULL and invalid hamming-index triggers as separate bugs/tests. They should not block the targeted v0.1.6 fix, but they are useful acceptance cases for the containment work.