[YSQL] Vector index placeholder costing

[tanujnay112]

Jira Link: DB-15959

Description

Currently the costing function ybvectorcostestimate just returns without doing anything. It leaves its parameters untouched which has its caller here dealing with uninitialized data, leading to potentially undefined behavior during planning. This is reproducible on an LTO build on the latest master.

create extension vector;
CREATE TABLE public.pg_vector_collection (
    id bigint PRIMARY KEY,
    embedding vector(3)
);

CREATE INDEX pgvector_index 
ON public.pg_vector_collection USING ybhnsw (embedding vector_cosine_ops);

explain select * from pg_vector_collection order by embedding <=> '[1,1,1]';
                                   QUERY PLAN                                   
--------------------------------------------------------------------------------
 Sort  (cost=152.33..154.83 rows=1000 width=48)
   Sort Key: ((embedding <=> '[1,1,1]'::vector))
   ->  Seq Scan on pg_vector_collection  (cost=0.00..102.50 rows=1000 width=48)
(3 rows)

explain select * from pg_vector_collection order by embedding <=> '[1,1,1]';
                                          QUERY PLAN                                           
-----------------------------------------------------------------------------------------------
 Index Scan using pgvector_index on pg_vector_collection  (cost=0.00..6.51 rows=1000 width=48)
   Order By: (embedding <=> '[1,1,1]'::vector)
(2 rows)

Debugging this made it appear on the sequential scan run that the index scan was costed at more than 10e200.

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.