Skip to content

Commit 2468c32

Browse files
committed
test(gooddata-pandas): Arrow path ground truth and parity coverage
Backfill column_totals_indexes into all 36 fixture meta.json files; extend parity tests to cover all four DataFrameMetadata fields (row_totals_indexes, column_totals_indexes, primary_labels_from_index, primary_labels_from_columns) and expand for_arrow_table tests from 4 hand-picked cases to the full fixture set. risk: nonprod
1 parent 251c36a commit 2468c32

4 files changed

Lines changed: 142 additions & 135 deletions

File tree

packages/gooddata-pandas/src/gooddata_pandas/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# (C) 2021 GoodData Corporation
22

33
from gooddata_pandas._version import __version__
4-
from gooddata_pandas.arrow_types import TypesMapper
4+
from gooddata_pandas.arrow_types import ArrowConfig, TypesMapper
55

66
try:
77
from gooddata_pandas.arrow_convertor import convert_arrow_table_to_dataframe

packages/gooddata-pandas/src/gooddata_pandas/arrow_types.py

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# (C) 2026 GoodData Corporation
22
from __future__ import annotations
33

4+
from dataclasses import dataclass, field
45
from enum import Enum
56

67

@@ -19,3 +20,35 @@ class TypesMapper(Enum):
1920
DEFAULT = "default"
2021
ARROW_STRINGS = "arrow_strings"
2122
CUSTOM = "custom"
23+
24+
25+
@dataclass
26+
class ArrowConfig:
27+
"""
28+
Arrow IPC conversion configuration for DataFrameFactory.
29+
30+
Controls *how* Arrow data is converted to a pandas DataFrame. Whether to
31+
use the Arrow path at all is set via the ``use_arrow`` parameter on
32+
``DataFrameFactory`` itself.
33+
34+
Set once on the factory; applies to every Arrow-path call (for_exec_def,
35+
for_exec_def_arrow, for_arrow_table, for_exec_result_id).
36+
37+
Attributes:
38+
self_destruct: When True, Arrow buffers are freed during conversion,
39+
reducing peak native memory at the cost of not being able to reuse
40+
the table after the call. Defaults to False.
41+
types_mapper: Controls how Arrow types are mapped to pandas dtypes.
42+
TypesMapper.DEFAULT (default) — no mapping; float64 and object strings,
43+
identical to the JSON execution path.
44+
TypesMapper.ARROW_STRINGS — strings use Arrow-backed StringDtype;
45+
all numeric types unchanged.
46+
TypesMapper.CUSTOM — uses custom_mapping; raises ValueError if
47+
custom_mapping is not provided.
48+
custom_mapping: Arrow type → pandas dtype mapping dict. Only used when
49+
types_mapper=TypesMapper.CUSTOM, ignored otherwise.
50+
"""
51+
52+
self_destruct: bool = False
53+
types_mapper: TypesMapper = TypesMapper.DEFAULT
54+
custom_mapping: dict | None = field(default=None)

0 commit comments

Comments
 (0)