Skip to content
This repository was archived by the owner on Apr 1, 2026. It is now read-only.

feat: support full round-trip persistence for multimodal reference cols#2511

Merged
shuoweil merged 9 commits intomainfrom
shuowei-feat-persist-obj-ref
Mar 18, 2026
Merged

feat: support full round-trip persistence for multimodal reference cols#2511
shuoweil merged 9 commits intomainfrom
shuowei-feat-persist-obj-ref

Conversation

@shuoweil
Copy link
Copy Markdown
Contributor

@shuoweil shuoweil commented Mar 12, 2026

Saves multimodal metadata descriptions on .to_gbq().

Fixes #<452681068> 🦕

@shuoweil shuoweil self-assigned this Mar 12, 2026
@shuoweil shuoweil requested review from a team as code owners March 12, 2026 02:48
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Mar 12, 2026
chelsea-lin
chelsea-lin reviewed Mar 12, 2026
View reviewed changes
bigframes/dtypes.py Outdated
) -> typing.Tuple[str, Dtype]:
is_repeated = field.mode == "REPEATED"
if field.field_type == "RECORD":
if field.description == "bigframes_dtype: OBJ_REF_DTYPE":
Copy link
Copy Markdown
Contributor

@chelsea-lin chelsea-lin Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, we can changes both convert_to_schema_field and convert_schema_field to achieve round-trip persistence without changes in dataframe.py.
Please check how TIMEDELTA_DESCRIPTION_TAG works.

Copy link
Copy Markdown
Contributor Author

@shuoweil shuoweil Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks for the tip! I've updated convert_to_schema_field and convert_schema_field to handle description tags for OBJ_REF_DTYPE and reverted the dataframe.py updates.

@shuoweil shuoweil requested a review from chelsea-lin March 16, 2026 21:59
@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: m Pull request size is medium. labels Mar 16, 2026
@shuoweil
Copy link
Copy Markdown
Contributor Author

shuoweil commented Mar 17, 2026 •
edited
Loading

Upon checking, the failed e2e and sample tests are not related to my code change:
=========================== short test summary info ============================
FAILED tests/system/large/functions/test_remote_function.py::test_remote_function_via_session_vpc
FAILED tests/system/large/functions/test_remote_function.py::test_remote_function_via_session_custom_sa
FAILED tests/system/large/functions/test_remote_function.py::test_remote_function_via_session_custom_build_sa[fully-qualified-sa]
FAILED tests/system/large/functions/test_remote_function.py::test_remote_function_via_session_custom_build_sa[just-sa-email]

Samples - Python 3.10, 3.11, 3.12
FAILED multimodal_test.py::test_multimodal_example

@shuoweil shuoweil force-pushed the shuowei-feat-persist-obj-ref branch from 80f64f5 to 2f67d9c Compare March 17, 2026 18:29
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Mar 17, 2026
@shuoweil shuoweil enabled auto-merge (squash) March 18, 2026 20:15
@shuoweil shuoweil merged commit 494a0a1 into main Mar 18, 2026
21 of 25 checks passed
@shuoweil shuoweil deleted the shuowei-feat-persist-obj-ref branch March 18, 2026 20:16
@sycai sycai mentioned this pull request Mar 31, 2026
Merged
sycai added a commit that referenced this pull request Apr 1, 2026
@sycai
chore: librarian release pull request: 20260331T225508Z (#2547)
be33279 
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.7.0
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:160860d189ff1c2f7515638478823712fa5b243e27ccc33a2728669fa1e2ed0c
<details><summary>bigframes: 2.39.0</summary>

##
[2.39.0](v2.38.0...v2.39.0)
(2026-03-31)

### Features

* add `df.bigquery.ai.forecast` method to pandas dataframe accessor
(#2518)
([1126cec](1126cec9))

* support full round-trip persistence for multimodal reference cols
(#2511)
([494a0a1](494a0a11))

* expose DataFrame.bigquery in both pandas and bigframes DataFrames
(#2533)
([69fe317](69fe3176))

### Bug Fixes

* to_gbq may swap data columns when replace table (#2532)
([17ecc65](17ecc65e))

* handle aggregate operations on empty selections (#2510)
([34fb5da](34fb5daa))

* Localize BigQuery log suppression for gbq.py (#2541)
([af49ca2](af49ca29))

* Respect remote function config changes even if logic unchanged (#2512)
([b952428](b9524284))

* support melting empty DataFrames without crashing (#2509)
([e8c4603](e8c46032))

### Performance Improvements

* Make executor data uploads async internally (#2529)
([96597f0](96597f0b))

### Documentation

* gemini retouch of the index page for seo (#2514)
([2e5311e](2e5311e2))

* Rename Blob column references to ObjectRef column (#2535)
([44e0ffd](44e0ffd9))

</details>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@TrevorBergeron TrevorBergeron Awaiting requested review from TrevorBergeron

1 more reviewer

@chelsea-lin chelsea-lin chelsea-lin approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@shuoweil shuoweil

Labels

api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@shuoweil @chelsea-lin