Skip to content

Commit 04aa5d1

Browse files
committed
Merge release-candidate/2025-10 in prepartion for v8.0 release
⚠️ **Python 3.9 is no longer supported.** The SDK now requires Python 3.10 or later. Python 3.9 reached end-of-life on October 2, 2025. Users must upgrade to Python 3.10+ to continue using the SDK. ⚠️ **Namespace parameter default behavior changed.** The SDK no longer applies default values for the `namespace` parameter in GRPC methods. When `namespace=None`, the parameter is omitted from requests, allowing the API to handle namespace defaults appropriately. This change affects `upsert_from_dataframe` methods in GRPC clients. The API is moving toward `"__default__"` as the default namespace value, and this change ensures the SDK doesn't override API defaults. Note: The official SDK package was renamed last year from `pinecone-client` to `pinecone` beginning in version 5.1.0. Please remove `pinecone-client` from your project dependencies and add `pinecone` instead to get the latest updates if upgrading from earlier versions. You can now configure dedicated read nodes for your serverless indexes, giving you more control over query performance and capacity planning. By default, serverless indexes use OnDemand read capacity, which automatically scales based on demand. With dedicated read capacity, you can allocate specific read nodes with manual scaling control. **Create an index with dedicated read capacity:** ```python from pinecone import ( Pinecone, ServerlessSpec, CloudProvider, AwsRegion, Metric ) pc = Pinecone() pc.create_index( name='my-index', dimension=1536, metric=Metric.COSINE, spec=ServerlessSpec( cloud=CloudProvider.AWS, region=AwsRegion.US_EAST_1, read_capacity={ "mode": "Dedicated", "dedicated": { "node_type": "t1", "scaling": "Manual", "manual": { "shards": 2, "replicas": 2 } } } ) ) ``` **Configure read capacity on an existing index:** You can switch between OnDemand and Dedicated modes, or adjust the number of shards and replicas for dedicated read capacity: ```python from pinecone import Pinecone pc = Pinecone() pc.configure_index( name='my-index', read_capacity={"mode": "OnDemand"} ) pc.configure_index( name='my-index', read_capacity={ "mode": "Dedicated", "dedicated": { "node_type": "t1", "scaling": "Manual", "manual": { "shards": 3, "replicas": 2 } } } ) pc.configure_index( name='my-index', read_capacity={ "mode": "Dedicated", "dedicated": { "node_type": "t1", "scaling": "Manual", "manual": { "shards": 4, "replicas": 3 } } } ) ``` When you change read capacity configuration, the index will transition to the new configuration. You can use `describe_index` to check the status of the transition. See [PR #528](#528) for details. You can now fetch vectors using metadata filters instead of vector IDs. This is especially useful when you need to retrieve vectors based on their metadata properties. ```python from pinecone import Pinecone pc = Pinecone() index = pc.Index(host="your-index-host") response = index.fetch_by_metadata( filter={'genre': {'$in': ['comedy', 'drama']}, 'year': {'$eq': 2019}}, namespace='my_namespace', limit=50 ) print(f"Found {len(response.vectors)} vectors") for vec_id, vector in response.vectors.items(): print(f"ID: {vec_id}, Metadata: {vector.metadata}") ``` **Pagination support:** When fetching large numbers of vectors, you can use pagination tokens to retrieve results in batches: ```python response = index.fetch_by_metadata( filter={'status': 'active'}, limit=100 ) if response.pagination and response.pagination.next: next_response = index.fetch_by_metadata( filter={'status': 'active'}, pagination_token=response.pagination.next, limit=100 ) ``` The update method used to require a vector id to be passed, but now you have the option to pass a metadata filter instead. This is useful for bulk metadata updates across many vectors. There is also a dry_run option that allows you to preview the number of vectors that would be changed by the update before performing the operation. ```python from pinecone import Pinecone pc = Pinecone() index = pc.Index(host="your-index-host") response = index.update( set_metadata={'status': 'active'}, filter={'genre': {'$eq': 'drama'}}, dry_run=True ) print(f"Would update {response.matched_records} vectors") response = index.update( set_metadata={'status': 'active'}, filter={'genre': {'$eq': 'drama'}} ) ``` A new `FilterBuilder` utility class provides a type-safe, fluent interface for constructing metadata filters. While perhaps a bit verbose, it can help prevent common errors like misspelled operator names and provides better IDE support. When you chain `.build()` onto the `FilterBuilder` it will emit a python dictionary representing the filter. Methods that take metadata filters as arguments will continue to accept dictionaries as before. ```python from pinecone import Pinecone, FilterBuilder pc = Pinecone() index = pc.Index(host="your-index-host") filter1 = FilterBuilder().eq("genre", "drama").build() filter2 = (FilterBuilder().eq("genre", "drama") & FilterBuilder().gt("year", 2020)).build() filter3 = (FilterBuilder().eq("genre", "comedy") | FilterBuilder().eq("genre", "drama")).build() filter4 = ((FilterBuilder().eq("genre", "drama") & FilterBuilder().gte("year", 2020)) | (FilterBuilder().eq("genre", "comedy") & FilterBuilder().lt("year", 2000))).build() response = index.fetch_by_metadata(filter=filter2, limit=50) index.update( set_metadata={'status': 'archived'}, filter=filter3 ) ``` The FilterBuilder supports all Pinecone filter operators: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in_`, `nin`, and `exists`. Compound expressions are build with and `&` and or `|`. See [PR #529](#529) for `fetch_by_metadata`, [PR #544](#544) for `update()` with filter, and [PR #531](#531) for FilterBuilder. You can now create namespaces in serverless indexes directly from the SDK: ```python from pinecone import Pinecone pc = Pinecone() index = pc.Index(host="your-index-host") namespace = index.create_namespace(name="my-namespace") print(f"Created namespace: {namespace.name}, Vector count: {namespace.vector_count}") namespace = index.create_namespace( name="my-namespace", schema={ "fields": { "genre": {"filterable": True}, "year": {"filterable": True} } } ) ``` **Note:** This operation is not supported for pod-based indexes. See [PR #532](#532) for details. For sparse indexes with integrated embedding configured to use the `pinecone-sparse-english-v0` model, you can now specify which terms must be present in search results: ```python from pinecone import Pinecone, SearchQuery pc = Pinecone() index = pc.Index(host="your-index-host") response = index.search( namespace="my-namespace", query=SearchQuery( inputs={"text": "Apple corporation"}, top_k=10, match_terms={ "strategy": "all", "terms": ["apple", "corporation"] } ) ) ``` The `match_terms` parameter ensures that all specified terms must be present in the text of each search hit. Terms are normalized and tokenized before matching, and order does not matter. See [PR #530](#530) for details. **Update API keys, projects, and organizations:** ```python from pinecone import Admin admin = Admin() # Auth with PINECONE_CLIENT_ID and PINECONE_CLIENT_SECRET api_key = admin.api_key.update( api_key_id='my-api-key-id', name='updated-api-key-name', roles=['ProjectEditor', 'DataPlaneEditor'] ) project = admin.project.update( project_id='my-project-id', name='updated-project-name', max_pods=10, force_encryption_with_cmek=True ) organization = admin.organization.update( organization_id='my-org-id', name='updated-organization-name' ) ``` **Delete organizations:** ```python from pinecone import Admin admin = Admin() admin.organization.delete(organization_id='my-org-id') ``` See [PR #527](#527) and [PR #543](#543) for details. You can now configure which metadata fields are filterable when creating serverless indexes. This helps optimize performance by only indexing metadata fields that you plan to use for filtering: ```python from pinecone import ( Pinecone, ServerlessSpec, CloudProvider, AwsRegion, Metric ) pc = Pinecone() pc.create_index( name='my-index', dimension=1536, metric=Metric.COSINE, spec=ServerlessSpec( cloud=CloudProvider.AWS, region=AwsRegion.US_EAST_1, schema={ "genre": {"filterable": True}, "year": {"filterable": True}, "rating": {"filterable": True} } ) ) ``` When using schemas, only fields marked as `filterable: True` in the schema can be used in metadata filters. See [PR #528](#528) for details. The SDK now exposes header information from API responses. This information is available in response objects via the `_response_info` attribute and can be useful for debugging and monitoring. ```python from pinecone import Pinecone pc = Pinecone() index = pc.Index(host="your-index-host") response = index.query( vector=[0.1, 0.2, 0.3, ...], top_k=10, namespace='my_namespace' ) for k, v in response._response_info.get('raw_headers').items(): print(f"{k}: {v}") ``` See [PR #539](#539) for details. We've replaced Python's standard library `json` module with `orjson`, a fast JSON library written in Rust. This provides significant performance improvements for both serialization and deserialization of request payloads: - **Serialization (dumps)**: 10-23x faster depending on payload size - **Deserialization (loads)**: 4-7x faster depending on payload size These improvements are especially beneficial for: - High-throughput applications making many API calls - Applications handling large vector payloads - Real-time applications where latency matters No code changes are required - the API remains the same, and you'll automatically benefit from these performance improvements. See [PR #556](#556) for details. We've optimized gRPC response parsing by replacing `json_format.MessageToDict` with direct protobuf field access. This optimization provides approximately 2x faster response parsing for gRPC operations. Special thanks to [@yorickvP](https://github.com/yorickvP) for surfacing the `json_format.MessageToDict` refactor opportunity. While we didn't merge the specific PR, yorick's insight led us to implement a similar optimization that significantly improves gRPC performance. See [PR #553](#553) for details. - **Type hints and IDE support**: Comprehensive type hints throughout the SDK improve IDE autocomplete and type checking. The SDK now uses Python 3.10+ type syntax throughout. - **Documentation**: Updated docstrings with RST formatting and code examples for better developer experience. - **Dependency updates**: Updated protobuf to 5.29.5 to address security vulnerabilities. Updated `pinecone-plugin-assistant` to version 3.0.1. - **Build system**: Migrated from poetry to uv for faster dependency management. - [@yorickvP](https://github.com/yorickvP) - Thanks for surfacing the gRPC response parsing optimization opportunity!
2 parents 6713a6f + 47c4cfa commit 04aa5d1

File tree

544 files changed

+39823
-10841
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

544 files changed

+39823
-10841
lines changed

.durations_grpc

Lines changed: 3421 additions & 0 deletions
Large diffs are not rendered by default.

.durations_rest_asyncio

Lines changed: 167 additions & 0 deletions
Large diffs are not rendered by default.

.durations_rest_sync

Lines changed: 301 additions & 0 deletions
Large diffs are not rendered by default.

.github/actions/build-docs/action.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,8 @@ inputs:
88
runs:
99
using: 'composite'
1010
steps:
11-
- name: Setup Poetry
12-
uses: ./.github/actions/setup-poetry
11+
- name: Setup uv
12+
uses: ./.github/actions/setup-uv
1313
with:
1414
include_grpc: 'true'
1515
include_dev: 'true'
@@ -19,4 +19,4 @@ runs:
1919
- name: Build html documentation
2020
shell: bash
2121
run: |
22-
poetry run sphinx-build -b html docs docsbuild
22+
uv run sphinx-build -b html docs docsbuild

.github/actions/cleanup-all/action.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,11 @@ inputs:
1919
runs:
2020
using: 'composite'
2121
steps:
22-
- name: Setup Poetry
23-
uses: ./.github/actions/setup-poetry
22+
- name: Setup uv
23+
uses: ./.github/actions/setup-uv
2424
- name: Cleanup all
2525
shell: bash
26-
run: poetry run python3 ./.github/actions/cleanup-all/cleanup-test-projects.py
26+
run: uv run python3 ./.github/actions/cleanup-all/cleanup-test-projects.py
2727
env:
2828
PINECONE_API_KEY: ${{ inputs.PINECONE_API_KEY }}
2929
PINECONE_ADDITIONAL_HEADERS: ${{ inputs.PINECONE_ADDITIONAL_HEADERS }}

.github/actions/index-create/action.yml

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,15 @@ inputs:
1717
dimension:
1818
description: 'The dimension of the index'
1919
required: false
20-
default: '3'
20+
default: ''
2121
metric:
2222
description: 'The metric of the index'
2323
required: false
2424
default: 'cosine'
25+
vector_type:
26+
description: 'The type of the index'
27+
required: false
28+
default: 'dense'
2529
PINECONE_API_KEY:
2630
description: 'The Pinecone API key'
2731
required: true
@@ -36,21 +40,26 @@ outputs:
3640
description: 'The name of the index, including randomized suffix'
3741
value: ${{ steps.create-index.outputs.index_name }}
3842

43+
index_host:
44+
description: 'The host of the index'
45+
value: ${{ steps.create-index.outputs.index_host }}
46+
3947
runs:
4048
using: 'composite'
4149
steps:
42-
- name: Setup Poetry
43-
uses: ./.github/actions/setup-poetry
50+
- name: Setup uv
51+
uses: ./.github/actions/setup-uv
4452

4553
- name: Create index
4654
id: create-index
4755
shell: bash
48-
run: poetry run python3 ./.github/actions/index-create/create.py
56+
run: uv run python3 ./.github/actions/index-create/create.py
4957
env:
5058
PINECONE_API_KEY: ${{ inputs.PINECONE_API_KEY }}
5159
PINECONE_ADDITIONAL_HEADERS: ${{ inputs.PINECONE_ADDITIONAL_HEADERS }}
5260
NAME_PREFIX: ${{ inputs.name_prefix }}
5361
REGION: ${{ inputs.region }}
5462
CLOUD: ${{ inputs.cloud }}
63+
VECTOR_TYPE: ${{ inputs.vector_type }}
5564
DIMENSION: ${{ inputs.dimension }}
5665
METRIC: ${{ inputs.metric }}

.github/actions/index-create/create.py

Lines changed: 30 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
import os
2-
import re
32
import random
43
import string
5-
from datetime import datetime
4+
import uuid
65
from pinecone import Pinecone
6+
from datetime import datetime
77

88

99
def read_env_var(name):
@@ -22,39 +22,9 @@ def write_gh_output(name, value):
2222
print(f"{name}={value}", file=fh)
2323

2424

25-
def generate_index_name(test_name: str) -> str:
26-
github_actor = os.getenv("GITHUB_ACTOR", None)
27-
user = os.getenv("USER", None)
28-
index_owner = github_actor or user
29-
30-
formatted_date = datetime.now().strftime("%Y%m%d-%H%M%S%f")[:-3]
31-
32-
github_job = os.getenv("GITHUB_JOB", None)
33-
34-
if test_name.startswith("test_"):
35-
test_name = test_name[5:]
36-
37-
# Remove trailing underscore, if any
38-
if test_name.endswith("_"):
39-
test_name = test_name[:-1]
40-
41-
name_parts = [index_owner, formatted_date, github_job, test_name]
42-
index_name = "-".join([x for x in name_parts if x is not None])
43-
44-
# Remove invalid characters
45-
replace_with_hyphen = re.compile(r"[\[\(_,\s]")
46-
index_name = re.sub(replace_with_hyphen, "-", index_name)
47-
replace_with_empty = re.compile(r"[\]\)\.]")
48-
index_name = re.sub(replace_with_empty, "", index_name)
49-
50-
max_length = 45
51-
index_name = index_name[:max_length]
52-
53-
# Trim final character if it is not alphanumeric
54-
if index_name.endswith("_") or index_name.endswith("-"):
55-
index_name = index_name[:-1]
56-
57-
return index_name.lower()
25+
def generate_index_name(name_prefix: str) -> str:
26+
name = name_prefix.lower() + "-" + str(uuid.uuid4())
27+
return name[:45]
5828

5929

6030
def get_tags():
@@ -74,15 +44,35 @@ def get_tags():
7444

7545
def main():
7646
pc = Pinecone(api_key=read_env_var("PINECONE_API_KEY"))
77-
index_name = generate_index_name(read_env_var("NAME_PREFIX") + random_string(20))
47+
index_name = generate_index_name(read_env_var("NAME_PREFIX"))
48+
dimension_var = read_env_var("DIMENSION")
49+
if dimension_var is not None and dimension_var != "":
50+
dimension = int(dimension_var)
51+
else:
52+
dimension = None
53+
54+
vector_type_var = read_env_var("VECTOR_TYPE")
55+
if vector_type_var is not None and vector_type_var != "":
56+
vector_type = vector_type_var
57+
else:
58+
vector_type = None
59+
60+
metric = read_env_var("METRIC")
61+
cloud = read_env_var("CLOUD")
62+
region = read_env_var("REGION")
63+
tags = get_tags()
64+
7865
pc.create_index(
7966
name=index_name,
80-
metric=read_env_var("METRIC"),
81-
dimension=int(read_env_var("DIMENSION")),
82-
spec={"serverless": {"cloud": read_env_var("CLOUD"), "region": read_env_var("REGION")}},
83-
tags=get_tags(),
67+
metric=metric,
68+
dimension=dimension,
69+
vector_type=vector_type,
70+
tags=tags,
71+
spec={"serverless": {"cloud": cloud, "region": region}},
8472
)
73+
description = pc.describe_index(name=index_name)
8574
write_gh_output("index_name", index_name)
75+
write_gh_output("index_host", description.host)
8676

8777

8878
if __name__ == "__main__":

.github/actions/index-delete/action.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,12 @@ inputs:
1717
runs:
1818
using: 'composite'
1919
steps:
20-
- name: Setup Poetry
21-
uses: ./.github/actions/setup-poetry
20+
- name: Setup uv
21+
uses: ./.github/actions/setup-uv
2222

2323
- name: Delete index
2424
shell: bash
25-
run: poetry run python3 ./.github/actions/index-delete/delete.py
25+
run: uv run python3 ./.github/actions/index-delete/delete.py
2626
env:
2727
PINECONE_API_KEY: ${{ inputs.PINECONE_API_KEY }}
2828
PINECONE_ADDITIONAL_HEADERS: ${{ inputs.PINECONE_ADDITIONAL_HEADERS }}

.github/actions/project-create/action.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ runs:
4040
- name: Set up Python
4141
uses: actions/setup-python@v5
4242
with:
43-
python-version: 3.9
43+
python-version: '3.10'
4444

4545
- name: Install deps
4646
shell: bash
@@ -50,7 +50,7 @@ runs:
5050
- name: Create project
5151
id: create-project
5252
shell: bash
53-
run: poetry run python3 ./.github/actions/project-create/script.py
53+
run: uv run python3 ./.github/actions/project-create/script.py
5454
env:
5555
API_VERSION: ${{ inputs.api_version }}
5656
PINECONE_SERVICE_ACCOUNT_CLIENT_ID: ${{ inputs.PINECONE_SERVICE_ACCOUNT_CLIENT_ID }}

.github/actions/project-delete/action.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ runs:
2828
- name: Set up Python
2929
uses: actions/setup-python@v5
3030
with:
31-
python-version: 3.9
31+
python-version: '3.10'
3232

3333
- name: Install deps
3434
shell: bash
@@ -45,7 +45,7 @@ runs:
4545
- name: Delete project
4646
id: delete-project
4747
shell: bash
48-
run: poetry run python3 ./.github/actions/project-delete/delete-project.py
48+
run: uv run python3 ./.github/actions/project-delete/delete-project.py
4949
env:
5050
API_VERSION: ${{ inputs.api_version }}
5151
PINECONE_SERVICE_ACCOUNT_CLIENT_ID: ${{ inputs.PINECONE_SERVICE_ACCOUNT_CLIENT_ID }}

0 commit comments

Comments
 (0)