Skip to content
This repository was archived by the owner on Apr 1, 2026. It is now read-only.

Commit e1576cc

Browse files
Merge branch 'main' into write_api_redux_redux
2 parents f7e85af + a6f499c commit e1576cc

File tree

554 files changed

+10620
-12349
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

554 files changed

+10620
-12349
lines changed

.gemini/common/constraints.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
## Constraints
2+
3+
- Only add git commits. Do not change git history.
4+
- Follow the spec file for development.
5+
- Check off items in the "Acceptance
6+
criteria" and "Detailed steps" sections with `[x]`.
7+
- Please do this as they are completed.
8+
- Refer back to the spec after each step.

.gemini/common/docs.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
## Documentation
2+
3+
If a method or property is implementing the same interface as a third-party
4+
package such as pandas or scikit-learn, place the relevant docstring in the
5+
corresponding `third_party/bigframes_vendored/package_name` directory, not in
6+
the `bigframes` directory. Implementations may be placed in the `bigframes`
7+
directory, though.
8+
9+
@../tools/test_docs.md

.gemini/tasks/scalar_op.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
## Adding a scalar operator
2+
3+
For an example, see commit
4+
[c5b7fdae74a22e581f7705bc0cf5390e928f4425](https://github.com/googleapis/python-bigquery-dataframes/commit/c5b7fdae74a22e581f7705bc0cf5390e928f4425).
5+
6+
To add a new scalar operator, follow these steps:
7+
8+
1. **Define the operation dataclass:**
9+
- In `bigframes/operations/`, find the relevant file (e.g., `geo_ops.py` for geography functions) or create a new one.
10+
- Create a new dataclass inheriting from `base_ops.UnaryOp` for unary
11+
operators, `base_ops.BinaryOp` for binary operators, `base_ops.TernaryOp`
12+
for ternary operators, or `base_ops.NaryOp for operators with many
13+
arguments. Note that these operators are counting the number column-like
14+
arguments. A function that takes only a single column but several literal
15+
values would still be a `UnaryOp`.
16+
- Define the `name` of the operation and any parameters it requires.
17+
- Implement the `output_type` method to specify the data type of the result.
18+
19+
2. **Export the new operation:**
20+
- In `bigframes/operations/__init__.py`, import your new operation dataclass and add it to the `__all__` list.
21+
22+
3. **Implement the user-facing function (pandas-like):**
23+
24+
- Identify the canonical function from pandas / geopandas / awkward array /
25+
other popular Python package that this operator implements.
26+
- Find the corresponding class in BigFrames. For example, the implementation
27+
for most geopandas.GeoSeries methods is in
28+
`bigframes/geopandas/geoseries.py`. Pandas Series methods are implemented
29+
in `bigframes/series.py` or one of the accessors, such as `StringMethods`
30+
in `bigframes/operations/strings.py`.
31+
- Create the user-facing function that will be called by users (e.g., `length`).
32+
- If the SQL method differs from pandas or geopandas in a way that can't be
33+
made the same, raise a `NotImplementedError` with an appropriate message and
34+
link to the feedback form.
35+
- Add the docstring to the corresponding file in
36+
`third_party/bigframes_vendored`, modeled after pandas / geopandas.
37+
38+
4. **Implement the user-facing function (SQL-like):**
39+
40+
- In `bigframes/bigquery/_operations/`, find the relevant file (e.g., `geo.py`) or create a new one.
41+
- Create the user-facing function that will be called by users (e.g., `st_length`).
42+
- This function should take a `Series` for any column-like inputs, plus any other parameters.
43+
- Inside the function, call `series._apply_unary_op`,
44+
`series._apply_binary_op`, or similar passing the operation dataclass you
45+
created.
46+
- Add a comprehensive docstring with examples.
47+
- In `bigframes/bigquery/__init__.py`, import your new user-facing function and add it to the `__all__` list.
48+
49+
5. **Implement the compilation logic:**
50+
- In `bigframes/core/compile/scalar_op_compiler.py`:
51+
- If the BigQuery function has a direct equivalent in Ibis, you can often reuse an existing Ibis method.
52+
- If not, define a new Ibis UDF using `@ibis_udf.scalar.builtin` to map to the specific BigQuery function signature.
53+
- Create a new compiler implementation function (e.g., `geo_length_op_impl`).
54+
- Register this function to your operation dataclass using `@scalar_op_compiler.register_unary_op` or `@scalar_op_compiler.register_binary_op`.
55+
- This implementation will translate the BigQuery DataFrames operation into the appropriate Ibis expression.
56+
57+
6. **Add Tests:**
58+
- Add system tests in the `tests/system/` directory to verify the end-to-end
59+
functionality of the new operator. Test various inputs, including edge cases
60+
and `NULL` values.
61+
62+
Where possible, run the same test code against pandas or GeoPandas and
63+
compare that the outputs are the same (except for dtypes if BigFrames
64+
differs from pandas).
65+
- If you are overriding a pandas or GeoPandas property, add a unit test to
66+
ensure the correct behavior (e.g., raising `NotImplementedError` if the
67+
functionality is not supported).

.gemini/tools/style_nox.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
## Code Style with nox
2+
3+
- We use the automatic code formatter `black`. You can run it using
4+
the nox session `format`. This will eliminate many lint errors. Run via:
5+
6+
```bash
7+
nox -r -s format
8+
```
9+
10+
- PEP8 compliance is required, with exceptions defined in the linter configuration.
11+
If you have ``nox`` installed, you can test that you have not introduced
12+
any non-compliant code via:
13+
14+
```
15+
nox -r -s lint
16+
```
17+
18+
- When writing tests, use the idiomatic "pytest" style.

.gemini/tools/test_docs.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
## Testing code samples
2+
3+
Code samples are very important for accurate documentation. We use the "doctest"
4+
framework to ensure the samples are functioning as expected. After adding a code
5+
sample, please ensure it is correct by running doctest. To run the samples
6+
doctests for just a single method, refer to the following example:
7+
8+
```bash
9+
pytest --doctest-modules bigframes/pandas/__init__.py::bigframes.pandas.cut
10+
```

.gemini/tools/test_nox.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
## Testing with nox
2+
3+
Use `nox` to instrument our tests.
4+
5+
- To test your changes, run unit tests with `nox`:
6+
7+
```bash
8+
nox -r -s unit
9+
```
10+
11+
- To run a single unit test:
12+
13+
```bash
14+
nox -r -s unit-3.14 -- -k <name of test>
15+
```
16+
17+
- Ignore this step if you lack access to Google Cloud resources. To run system
18+
tests, you can execute::
19+
20+
# Run all system tests
21+
$ nox -r -s system
22+
23+
# Run a single system test
24+
$ nox -r -s system-3.14 -- -k <name of test>
25+
26+
- The codebase must have better coverage than it had previously after each
27+
change. You can test coverage via `nox -s unit system cover` (takes a long
28+
time). Omit `system` if you lack access to cloud resources.

.gemini/tools/test_pytest.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
## Testing with pytest
2+
3+
Use `pytest` to instrument our tests.
4+
5+
- To test your changes, run `pytest`:
6+
7+
```bash
8+
pytest <test_file>::<test_case>
9+
```

.github/CODEOWNERS

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@
55
# https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners#codeowners-syntax
66
# Note: This file is autogenerated. To make changes to the codeowner team, please update .repo-metadata.json.
77

8-
# @googleapis/yoshi-python @googleapis/api-bigquery-dataframe are the default owners for changes in this repo
9-
* @googleapis/yoshi-python @googleapis/api-bigquery-dataframe
8+
# @googleapis/cloud-sdk-python-team @googleapis/bigquery-dataframe-team are the default owners for changes in this repo
9+
* @googleapis/cloud-sdk-python-team @googleapis/bigquery-dataframe-team
1010

11-
# @googleapis/python-samples-reviewers @googleapis/api-bigquery-dataframe are the default owners for samples changes
12-
/samples/ @googleapis/python-samples-reviewers @googleapis/api-bigquery-dataframe
11+
# @googleapis/python-samples-reviewers @googleapis/bigquery-dataframe-team are the default owners for samples changes
12+
/samples/ @googleapis/python-samples-reviewers @googleapis/bigquery-dataframe-team

.github/blunderbuss.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,14 @@
44
# Note: This file is autogenerated. To make changes to the assignee
55
# team, please update `codeowner_team` in `.repo-metadata.json`.
66
assign_issues:
7-
- googleapis/api-bigquery-dataframe
7+
- googleapis/bigquery-dataframe-team
88

99
assign_issues_by:
1010
- labels:
1111
- "samples"
1212
to:
1313
- googleapis/python-samples-reviewers
14-
- googleapis/api-bigquery-dataframe
14+
- googleapis/bigquery-dataframe-team
1515

1616
assign_prs:
17-
- googleapis/api-bigquery-dataframe
17+
- googleapis/bigquery-dataframe-team

.github/workflows/lint.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515
- name: Setup Python
1616
uses: actions/setup-python@v5
1717
with:
18-
python-version: "3.10"
18+
python-version: "3.14"
1919
- name: Install nox
2020
run: |
2121
python -m pip install --upgrade setuptools pip wheel

0 commit comments

Comments
 (0)